-- Logs begin at Wed 2022-07-13 22:23:37 UTC, end at Wed 2022-07-13 23:39:32 UTC. -- Jul 13 23:00:17.612121 dv-cinder-799871-29-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 13 23:00:17.623075 dv-cinder-799871-29-iscsi devstack@keystone.service[80272]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 13 23:00:17.625180 dv-cinder-799871-29-iscsi devstack@keystone.service[80272]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jul 13 23:00:17 2022] *** Jul 13 23:00:17.625180 dv-cinder-799871-29-iscsi devstack@keystone.service[80272]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 13 23:00:17.625180 dv-cinder-799871-29-iscsi devstack@keystone.service[80272]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 13 23:00:17.625180 dv-cinder-799871-29-iscsi devstack@keystone.service[80272]: nodename: dv-cinder-799871-29-iscsi Jul 13 23:00:17.625180 dv-cinder-799871-29-iscsi devstack@keystone.service[80272]: machine: x86_64 Jul 13 23:00:17.625180 dv-cinder-799871-29-iscsi devstack@keystone.service[80272]: clock source: unix Jul 13 23:00:17.625180 dv-cinder-799871-29-iscsi devstack@keystone.service[80272]: pcre jit disabled Jul 13 23:00:17.625180 dv-cinder-799871-29-iscsi devstack@keystone.service[80272]: detected number of CPU cores: 4 Jul 13 23:00:17.625180 dv-cinder-799871-29-iscsi devstack@keystone.service[80272]: current working directory: / Jul 13 23:00:17.625180 dv-cinder-799871-29-iscsi devstack@keystone.service[80272]: detected binary path: /usr/bin/uwsgi-core Jul 13 23:00:17.625180 dv-cinder-799871-29-iscsi devstack@keystone.service[80272]: your processes number limit is 47680 Jul 13 23:00:17.625180 dv-cinder-799871-29-iscsi devstack@keystone.service[80272]: your memory page size is 4096 bytes Jul 13 23:00:17.625790 dv-cinder-799871-29-iscsi devstack@keystone.service[80272]: detected max file descriptor number: 2048 Jul 13 23:00:17.625790 dv-cinder-799871-29-iscsi devstack@keystone.service[80272]: lock engine: pthread robust mutexes Jul 13 23:00:17.625790 dv-cinder-799871-29-iscsi devstack@keystone.service[80272]: thunder lock: enabled Jul 13 23:00:17.625790 dv-cinder-799871-29-iscsi devstack@keystone.service[80272]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 13 23:00:17.625790 dv-cinder-799871-29-iscsi devstack@keystone.service[80272]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 13 23:00:17.668902 dv-cinder-799871-29-iscsi devstack@keystone.service[80272]: Python main interpreter initialized at 0x56109000e9c0 Jul 13 23:00:17.668902 dv-cinder-799871-29-iscsi devstack@keystone.service[80272]: python threads support enabled Jul 13 23:00:17.668902 dv-cinder-799871-29-iscsi devstack@keystone.service[80272]: your server socket listen backlog is limited to 100 connections Jul 13 23:00:17.668902 dv-cinder-799871-29-iscsi devstack@keystone.service[80272]: your mercy for graceful operations on workers is 90 seconds Jul 13 23:00:17.669280 dv-cinder-799871-29-iscsi devstack@keystone.service[80272]: mapped 403077 bytes (393 KB) for 2 cores Jul 13 23:00:17.669358 dv-cinder-799871-29-iscsi devstack@keystone.service[80272]: *** Operational MODE: preforking *** Jul 13 23:00:17.669632 dv-cinder-799871-29-iscsi devstack@keystone.service[80272]: *** uWSGI is running in multiple interpreter mode *** Jul 13 23:00:17.669632 dv-cinder-799871-29-iscsi devstack@keystone.service[80272]: spawned uWSGI master process (pid: 80272) Jul 13 23:00:17.669847 dv-cinder-799871-29-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 13 23:00:17.674883 dv-cinder-799871-29-iscsi devstack@keystone.service[80272]: spawned uWSGI worker 1 (pid: 80284, cores: 1) Jul 13 23:00:17.674883 dv-cinder-799871-29-iscsi devstack@keystone.service[80272]: spawned uWSGI worker 2 (pid: 80285, cores: 1) Jul 13 23:00:17.674883 dv-cinder-799871-29-iscsi devstack@keystone.service[80272]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 13 23:00:23.150913 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 13 23:00:23.150913 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: from cryptography.utils import int_from_bytes Jul 13 23:00:23.151822 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 13 23:00:23.151822 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: from cryptography.utils import int_from_bytes Jul 13 23:00:23.154717 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 13 23:00:23.154717 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: from cryptography.utils import int_from_bytes Jul 13 23:00:23.155234 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 13 23:00:23.155234 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: from cryptography.utils import int_from_bytes Jul 13 23:00:23.201260 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 13 23:00:23.201443 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 13 23:00:23.201637 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] command line args: [] {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 13 23:00:23.201897 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 13 23:00:23.202076 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ================================================================================ {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 13 23:00:23.202331 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] admin_token = **** {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.202579 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] config_dir = [] {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.202766 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.202948 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] config_source = [] {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.203128 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] debug = True {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.203325 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.203713 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.203902 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.204153 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.204236 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 13 23:00:23.204398 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 13 23:00:23.204481 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.204623 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] command line args: [] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 13 23:00:23.204701 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] list_limit = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.204832 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] log_config_append = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.204915 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 13 23:00:23.204977 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.205045 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ================================================================================ {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 13 23:00:23.205201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] log_dir = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.205286 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] admin_token = **** {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.205353 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] log_file = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.205487 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] config_dir = [] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.205564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.205704 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.205839 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.205973 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] config_source = [] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.206100 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.206227 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] debug = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.206227 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.206517 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.206517 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.206768 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.206902 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.207046 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.207182 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.207311 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.207438 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.207587 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.207731 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.207872 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] list_limit = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.207872 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] log_config_append = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.208136 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.208271 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.208405 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.208532 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] log_dir = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.208669 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.208806 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] log_file = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.208944 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.209072 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.209204 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.209338 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.209338 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.209616 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.209747 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.209747 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.210003 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.60/identity {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.210139 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.210283 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] publish_errors = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.210418 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.210553 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.210697 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.210838 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.210974 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.211109 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.211286 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.211424 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.211592 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.211741 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.211870 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.211999 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] standard_threads = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.212126 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.212253 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.212385 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.212521 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.212521 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.212762 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.212908 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] use_journal = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.212908 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] use_json = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.212908 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] use_stderr = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.213281 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.60/identity {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.214666 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] use_syslog = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.214976 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] publish_errors = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.215173 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.215403 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.215618 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.215855 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.215855 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.216240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.216653 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.216947 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.216947 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.217577 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.217997 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.218410 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] standard_threads = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.218642 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.218995 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.219646 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.219801 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.219801 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.220180 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.220324 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] use_journal = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.220460 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.220588 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] use_json = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.220748 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.220882 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] use_stderr = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.220882 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] use_syslog = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.221140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.221288 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 23:00:23.221418 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.221546 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.221673 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.221813 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.221982 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.222144 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.222277 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.222412 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.222540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.222682 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.222811 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.222938 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.223076 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.223204 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.223331 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.223467 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.223642 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.223772 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.223899 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.224026 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.224153 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.224359 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.224490 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.224624 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.224787 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.224927 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.225065 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.225204 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.225339 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.225474 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.225610 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.225739 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.225866 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.226004 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.226133 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.226259 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.226386 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.226513 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.226640 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.226785 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.226913 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.227092 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.227230 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.227358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.227492 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.227666 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.227803 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.227931 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.228065 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.228193 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.228328 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.228463 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.228601 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.228729 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.228856 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.228989 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.229126 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.229254 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.229388 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.229522 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.229656 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.229785 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.229956 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.230116 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.230246 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.230373 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.230997 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.231133 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.231332 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.231463 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.231643 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.231774 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.231902 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.232030 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.232169 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.232304 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.232438 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.232567 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.232699 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.232828 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.233995 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.234166 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.234405 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.234569 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.234759 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.234955 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.235189 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.235354 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.235583 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.235747 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.235937 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.236128 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.236357 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.236530 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.236721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.236910 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.237100 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.237352 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.237558 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.237802 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.237991 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.238213 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.238424 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.238677 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.238925 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.239091 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.239267 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.239407 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.239597 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.239733 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.239903 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.240046 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.240176 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.240314 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.240444 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.240603 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.240736 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.240873 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.241002 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.241137 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.241278 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.241414 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.241543 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.241680 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.241809 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.241965 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.242108 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.242238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.242368 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.242496 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.242633 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.242762 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.242762 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.243013 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.243141 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.243277 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.243408 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.243552 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.243687 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.243816 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.243944 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.244092 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.244223 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.244363 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.244498 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.244684 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.244818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.244968 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.245099 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.245227 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.245361 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.245492 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.245627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.245765 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.245895 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.246024 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.246759 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.246894 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.247022 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.247160 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.247290 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.247429 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.247609 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.247745 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.247874 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.248080 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.248214 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.248342 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.248477 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.248608 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.248737 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.248878 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.249018 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.249159 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.249299 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.249428 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.249568 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.249700 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.249829 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.249966 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.250096 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.250225 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.250353 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.250481 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.250609 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.250747 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.250878 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.251006 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] auth.external = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.251144 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.251272 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.251400 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.251553 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.251691 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.251821 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.251950 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.252088 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] auth.password = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.252227 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.252433 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] auth.token = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.252578 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.252709 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.252838 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.252983 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.252983 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.253267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.253416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.253547 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.253682 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.253813 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.253944 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.254073 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] auth.external = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.254073 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.254326 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.254326 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] credential.caching = True {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.254570 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.254700 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.254838 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.254975 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.255106 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] auth.password = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.255241 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.255371 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] auth.token = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.255371 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.255675 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.255822 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.255954 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.256083 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.256212 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.256341 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.256470 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.256607 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.256748 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.256878 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.257007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.257137 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.257266 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.257410 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] credential.caching = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.257545 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.257674 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.257802 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.257931 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.258062 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.258199 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.258331 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.258477 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.258477 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.258732 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.258860 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.258997 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] federation.caching = True {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.258997 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.259249 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.259379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.259509 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.259677 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.259820 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.259951 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.260080 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.260216 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.260358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.260487 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.260624 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.260755 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.260884 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.261012 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.261140 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.261270 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.261408 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.261538 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.261730 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] federation.caching = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.261961 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.262125 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.262269 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] identity.caching = True {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.262408 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.262539 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.262668 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.262807 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.262937 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.263066 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.263249 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.263461 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.263681 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.263826 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.263958 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.264163 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.264297 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.264440 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.264570 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.264699 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.264829 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.264966 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.265097 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.265226 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.265354 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.caching = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.265484 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.265612 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.265750 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.265880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.266009 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.266147 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.266277 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.266405 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.266544 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.266674 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.266803 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.266932 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.267066 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.267197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.267334 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.267476 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.267630 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.267761 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.267906 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.268048 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.268185 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.268317 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.268447 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.268577 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.268706 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.268835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.268973 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.269109 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.269240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.269369 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.269498 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.269628 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.269774 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.269905 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.270035 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.270164 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.270293 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.270428 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.270567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.270699 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.270834 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.270963 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.271092 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.271220 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.271358 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.271488 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.271674 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.271806 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.271934 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.272064 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.272204 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.272336 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.272465 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.272594 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.272723 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.272853 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.272992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.273123 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.273252 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.273391 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.273520 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.273649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.273787 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.273918 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.274048 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.274048 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.274291 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.274420 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.274558 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.274558 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.274803 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.274933 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.275062 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.275199 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.275337 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.275467 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.275616 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.275747 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.275876 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.276005 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.276144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.user = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.276275 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.276403 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.276532 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.276660 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.276789 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.276931 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.277074 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.277203 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.277342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.277472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.277610 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.277740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.277868 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.277999 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.278128 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.278256 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.278394 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.278525 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.278653 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.278792 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.278922 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.279051 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.279188 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.279318 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.279447 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.279625 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.279760 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.279889 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.280030 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.280162 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.280291 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.280421 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.280550 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.280689 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.280826 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.280957 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.281086 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.281086 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.281338 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.281469 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.281607 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.281737 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.281865 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.281995 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.282123 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.282253 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.282390 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.282529 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.282658 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.282787 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.282916 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.283045 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.283182 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.283313 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.283442 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.283586 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.283720 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.283850 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.283990 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.284121 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.284260 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.284390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.284518 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.284647 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.284787 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.284917 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.285045 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.285259 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.285422 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.285555 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.285694 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.285824 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.285954 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.286083 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.286222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] resource.caching = True {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.286352 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.286490 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.286621 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.286750 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.286880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.287010 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.287139 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.287286 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.287418 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.287576 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.287713 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.287843 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.287972 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.288127 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.288259 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.288388 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.288519 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.288648 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.288776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] resource.caching = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.288914 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] role.caching = True {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.289045 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.289181 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] role.driver = sql {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.289316 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.289452 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.289587 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.289796 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.289933 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.290078 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.290214 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.290350 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.290488 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.290633 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.290769 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.290904 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.291050 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.291186 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.291321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.291464 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.291625 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] role.caching = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.291763 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.291907 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] role.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.292042 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.292188 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.292357 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.292495 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.292630 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.292767 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.292904 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.293681 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.293822 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.293972 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.293972 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.294234 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.294387 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.294532 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.294669 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.294805 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.294940 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.295075 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.295210 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.295353 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.295491 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.295649 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.295747 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.295826 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.295902 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.295996 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.296075 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.296148 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.296221 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.296294 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.296366 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.296448 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.296523 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.296597 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.296597 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.296726 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.296799 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.296880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.296955 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.297027 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.297027 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.297155 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.297155 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.297296 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.297369 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.297441 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.297512 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.297584 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.297664 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] token.caching = True {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.297739 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.297811 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.297884 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.297884 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.298003 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.298003 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.298143 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.298143 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.298273 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.298345 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.298417 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.298497 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.298571 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.298643 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.298714 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.298786 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.298857 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.298937 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] token.caching = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.299011 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.299083 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.299155 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.299226 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.299297 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.299383 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.299458 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.299550 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.299628 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.299700 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.299772 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.299856 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.299932 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.300009 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.300088 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.300161 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.300232 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.300312 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.300390 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] database.connection = **** {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.300472 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.300554 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.300629 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.300700 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.300772 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.300843 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.300923 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.300997 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.301069 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.301141 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.301211 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.301282 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.301361 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.301435 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.301513 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.301585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.301656 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.connection = **** {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.301739 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.301821 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.301902 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.301976 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.302048 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.302120 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.302192 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.302263 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.302342 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.302417 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.302490 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.302561 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.302632 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.302703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.302783 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.302857 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.302929 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.303001 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.303073 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.303073 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80285) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 13 23:00:23.303211 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.303211 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.303211 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.303211 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.303211 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.303211 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.303211 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.303211 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.303211 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 23:00:23.303757 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80284) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 13 23:00:23.320352 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.320578 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.320795 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.330233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80285) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.330463 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80285) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.330675 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80285) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.330880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.339234 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80285) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.363381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.363715 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.365840 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80285) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.367761 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80285) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.507631 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80285) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.508347 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.556510 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80285) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.556803 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80285) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.562319 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.562556 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.622810 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.623084 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.623176 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80285) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.623267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.623335 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80285) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.623627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.623718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80285) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.623782 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.623842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80285) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.623916 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.623981 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80285) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.624311 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80285) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.648599 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.648840 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.649069 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.649162 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80285) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.649338 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.649421 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80285) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.649585 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.649677 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80285) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.649743 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.649803 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80285) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.650037 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.650119 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80285) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.650182 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80285) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.650398 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80285) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.661787 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.662212 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80285) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.675186 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.675408 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.675509 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80285) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.675685 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.675751 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80285) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.675947 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.676028 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80285) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.676104 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.676169 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80285) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.676414 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.676496 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80285) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.676559 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80284) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.676619 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80285) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.676861 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80285) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 23:00:23.693923 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.694035 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.694225 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 13 23:00:23.694310 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 13 23:00:23.694506 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.694588 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.694652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 13 23:00:23.694854 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 13 23:00:23.695163 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.695249 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.695426 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.695509 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.695742 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.695824 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.695987 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.696080 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.707130 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.707225 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.707399 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.707481 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.707681 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.707778 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.707957 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.708039 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.716573 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 23:00:23.716738 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80285) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 23:00:23.719990 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80285) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 23:00:23.720178 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 23:00:23.720321 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.720496 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.720644 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.720828 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.720970 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.721109 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.721257 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.721397 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.721557 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.721736 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.721882 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.722022 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.722160 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.722299 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.722519 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.722665 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.722665 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.722928 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.722928 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.723209 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.750201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.751840 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80285) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 23:00:23.752117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.754067 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.755699 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 23:00:23.755977 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.757418 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80285) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 23:00:23.757744 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.757982 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.761220 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 23:00:23.761561 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.761864 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.765976 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80285) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 23:00:23.766242 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.770272 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 23:00:23.770569 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.770732 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80285) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 23:00:23.770992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.771275 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.772547 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.772547 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.772547 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.772547 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.773426 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.776114 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 23:00:23.776401 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.776683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.776962 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.777234 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.777528 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.777794 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.778058 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.792218 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.792495 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.796712 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.796986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.797388 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80285) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 23:00:23.801059 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.801356 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.801670 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 23:00:23.805297 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.805649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.806495 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80285) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 23:00:23.810662 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80285) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 23:00:23.811284 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 23:00:23.814338 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.814635 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.814894 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 23:00:23.815045 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.815187 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.815470 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.815783 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.822730 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.823029 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.823328 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.823644 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.823925 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.824203 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.838774 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80285) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 23:00:23.839070 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.839341 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.839638 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.844053 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 23:00:23.844336 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.844605 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.844874 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.847626 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.849335 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.849611 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.852796 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.852903 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80285) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 23:00:23.853087 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.853387 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.853652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.853931 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.854534 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.854804 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.857959 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 23:00:23.858237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.858637 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.858922 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.859256 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.867338 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80285) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 23:00:23.867639 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.867924 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.868216 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.868494 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.868782 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.869057 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.873096 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 23:00:23.873374 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.873655 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.873932 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.874208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.874500 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.874774 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.889407 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80285) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 23:00:23.892722 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80285) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 23:00:23.894650 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 23:00:23.896043 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.897665 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.897950 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 23:00:23.899440 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80285) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 23:00:23.899744 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.900033 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.901286 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.902899 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.904693 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 23:00:23.904973 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.905259 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.907732 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.909379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80285) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 23:00:23.912633 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.912723 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.912947 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.913235 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.913519 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.914242 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 23:00:23.917473 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.917760 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.919577 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.919577 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.922821 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80285) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 23:00:23.923103 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.923383 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.927549 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 23:00:23.927835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.928107 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.931310 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80285) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 23:00:23.931703 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.931990 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.932271 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.932537 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.932833 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.933087 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.933363 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.934246 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.934539 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.934812 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.935072 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.935336 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.935628 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80285) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.936448 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80284) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 23:00:23.936719 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.937001 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.937271 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.937542 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.937816 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.938082 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.938353 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.938630 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.939001 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.939281 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.939557 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.939816 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.940079 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80284) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 23:00:23.973844 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56109000e9c0 pid: 80285 (default app) Jul 13 23:00:23.977293 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce754996-f274-4f83-972c-e1b2c29abc20 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:23.977593 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce754996-f274-4f83-972c-e1b2c29abc20 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:23.977883 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce754996-f274-4f83-972c-e1b2c29abc20 None None] PATH_INFO: `/v3` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:23.978159 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56109000e9c0 pid: 80284 (default app) Jul 13 23:00:23.982677 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1/1] 10.222.0.60 () {50 vars in 791 bytes} [Wed Jul 13 23:00:23 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:00:31.564403 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281014dc-0369-4cba-adc8-15f0ddcc7b63 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:31.564969 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281014dc-0369-4cba-adc8-15f0ddcc7b63 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:31.564969 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281014dc-0369-4cba-adc8-15f0ddcc7b63 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:31.569356 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1/2] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:00:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:00:31.575392 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb521c7-3be2-4bf9-8a29-ab8035f1ef3c None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:31.575914 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb521c7-3be2-4bf9-8a29-ab8035f1ef3c None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:31.576005 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb521c7-3be2-4bf9-8a29-ab8035f1ef3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:31.644983 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: INFO dbcounter [None req-ddb521c7-3be2-4bf9-8a29-ab8035f1ef3c None None] Registered counter for database keystone Jul 13 23:00:31.666377 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG oslo_db.sqlalchemy.engines [None req-ddb521c7-3be2-4bf9-8a29-ab8035f1ef3c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80285) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 13 23:00:31.673292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writer thread running {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 13 23:00:31.855010 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG passlib.handlers.bcrypt [None req-ddb521c7-3be2-4bf9-8a29-ab8035f1ef3c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80285) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 13 23:00:31.855979 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG passlib.handlers.bcrypt [None req-ddb521c7-3be2-4bf9-8a29-ab8035f1ef3c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80285) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 13 23:00:31.911050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-ddb521c7-3be2-4bf9-8a29-ab8035f1ef3c None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:00:31.912674 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-ddb521c7-3be2-4bf9-8a29-ab8035f1ef3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:00:32.036211 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 2/3] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:00:31 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 462 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 13 23:00:32.239861 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281014dc-0369-4cba-adc8-15f0ddcc7b63 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:32.308937 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: INFO dbcounter [None req-281014dc-0369-4cba-adc8-15f0ddcc7b63 None None] Registered counter for database keystone Jul 13 23:00:32.328868 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG oslo_db.sqlalchemy.engines [None req-281014dc-0369-4cba-adc8-15f0ddcc7b63 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80284) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 13 23:00:32.335166 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writer thread running {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 13 23:00:32.454606 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281014dc-0369-4cba-adc8-15f0ddcc7b63 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:32.458900 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f33956c-51d5-405f-8571-3682cdca45cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:32.459618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33956c-51d5-405f-8571-3682cdca45cc admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:32.459903 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33956c-51d5-405f-8571-3682cdca45cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:32.460176 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33956c-51d5-405f-8571-3682cdca45cc admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:32.469817 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f33956c-51d5-405f-8571-3682cdca45cc admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:32.521905 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5f33956c-51d5-405f-8571-3682cdca45cc admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 13 23:00:32.522177 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f33956c-51d5-405f-8571-3682cdca45cc admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:32.533107 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 2/4] 10.222.0.60 () {58 vars in 1324 bytes} [Wed Jul 13 23:00:32 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 13 23:00:32.539042 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb521c7-3be2-4bf9-8a29-ab8035f1ef3c None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:32.553814 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb521c7-3be2-4bf9-8a29-ab8035f1ef3c None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:32.558295 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268a84a-62ec-402d-a131-8418334ef149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:32.559512 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268a84a-62ec-402d-a131-8418334ef149 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:32.559890 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268a84a-62ec-402d-a131-8418334ef149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:32.560233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268a84a-62ec-402d-a131-8418334ef149 admin admin] PATH_INFO: `/v3/projects/c5ffbcb80ba6476698f912924febd384` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:32.564512 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268a84a-62ec-402d-a131-8418334ef149 admin admin] RBAC: Authorizing `identity:get_project(project_id=c5ffbcb80ba6476698f912924febd384)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:32.615836 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e268a84a-62ec-402d-a131-8418334ef149 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 13 23:00:32.616416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268a84a-62ec-402d-a131-8418334ef149 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:32.621377 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 3/5] 10.222.0.60 () {56 vars in 1207 bytes} [Wed Jul 13 23:00:32 2022] GET /identity/v3/projects/c5ffbcb80ba6476698f912924febd384 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:00:34.628142 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ebeb92-1f23-4534-bdd0-74f9454fcd79 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:34.628662 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ebeb92-1f23-4534-bdd0-74f9454fcd79 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:34.628753 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ebeb92-1f23-4534-bdd0-74f9454fcd79 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:34.632820 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 3/6] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:00:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:00:34.638251 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef01f9a-d9dd-44f5-be34-9a3d4e4e2f65 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:34.638672 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef01f9a-d9dd-44f5-be34-9a3d4e4e2f65 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:34.638862 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef01f9a-d9dd-44f5-be34-9a3d4e4e2f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:34.657250 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-cef01f9a-d9dd-44f5-be34-9a3d4e4e2f65 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:00:34.658767 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-cef01f9a-d9dd-44f5-be34-9a3d4e4e2f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:00:34.666554 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 4/7] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:00:34 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 13 23:00:34.688946 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ebeb92-1f23-4534-bdd0-74f9454fcd79 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:34.701244 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ebeb92-1f23-4534-bdd0-74f9454fcd79 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:34.705435 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040e242d-442d-48e4-ab70-15be350a7d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:34.706061 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e242d-442d-48e4-ab70-15be350a7d6e admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:34.706344 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e242d-442d-48e4-ab70-15be350a7d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:34.706619 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e242d-442d-48e4-ab70-15be350a7d6e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:34.714223 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040e242d-442d-48e4-ab70-15be350a7d6e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:34.716032 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040e242d-442d-48e4-ab70-15be350a7d6e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:34.724739 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 4/8] 10.222.0.60 () {58 vars in 1324 bytes} [Wed Jul 13 23:00:34 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 23:00:34.730628 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef01f9a-d9dd-44f5-be34-9a3d4e4e2f65 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:34.739728 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef01f9a-d9dd-44f5-be34-9a3d4e4e2f65 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:34.744077 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9cae75-732b-4c36-8a62-1a0e57f0a838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:34.744941 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9cae75-732b-4c36-8a62-1a0e57f0a838 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:34.745229 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9cae75-732b-4c36-8a62-1a0e57f0a838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:34.745509 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9cae75-732b-4c36-8a62-1a0e57f0a838 admin admin] PATH_INFO: `/v3/users/7f1cf8ea49bb4e7fb83c77a4a463d6fd` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:34.747459 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9cae75-732b-4c36-8a62-1a0e57f0a838 admin admin] RBAC: Authorizing `identity:get_user(user_id=7f1cf8ea49bb4e7fb83c77a4a463d6fd)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:34.749325 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9cae75-732b-4c36-8a62-1a0e57f0a838 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:34.750909 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 5/9] 10.222.0.60 () {56 vars in 1198 bytes} [Wed Jul 13 23:00:34 2022] GET /identity/v3/users/7f1cf8ea49bb4e7fb83c77a4a463d6fd => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:00:36.840833 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc0145-0425-49d9-9551-a7180f47ae2f None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:36.841366 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc0145-0425-49d9-9551-a7180f47ae2f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:36.841366 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc0145-0425-49d9-9551-a7180f47ae2f None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:36.842919 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 5/10] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:00:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:00:36.847898 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0877d26-c27f-4589-a530-257c634f04e9 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:36.848324 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0877d26-c27f-4589-a530-257c634f04e9 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:36.848465 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0877d26-c27f-4589-a530-257c634f04e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:36.864724 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-b0877d26-c27f-4589-a530-257c634f04e9 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:00:36.865892 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-b0877d26-c27f-4589-a530-257c634f04e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:00:36.885748 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4828f88d-f687-4ed0-96ed-9b267497c64b None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:36.886038 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4828f88d-f687-4ed0-96ed-9b267497c64b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:36.886319 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4828f88d-f687-4ed0-96ed-9b267497c64b None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:36.888239 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 6/11] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:00:36.893076 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668ab7f-51ee-4864-9fa8-408c2beba9ab None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:36.893365 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668ab7f-51ee-4864-9fa8-408c2beba9ab None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:36.894170 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668ab7f-51ee-4864-9fa8-408c2beba9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:36.932831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG passlib.handlers.bcrypt [None req-f668ab7f-51ee-4864-9fa8-408c2beba9ab None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80284) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 13 23:00:36.933255 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG passlib.handlers.bcrypt [None req-f668ab7f-51ee-4864-9fa8-408c2beba9ab None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80284) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 13 23:00:36.948298 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 6/12] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:00:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 23:00:36.972429 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0877d26-c27f-4589-a530-257c634f04e9 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:36.979697 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-f668ab7f-51ee-4864-9fa8-408c2beba9ab None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:00:36.981423 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f668ab7f-51ee-4864-9fa8-408c2beba9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:00:36.995326 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:37.073594 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0877d26-c27f-4589-a530-257c634f04e9 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:37.078976 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 7/13] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:00:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 23:00:37.101235 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f668ab7f-51ee-4864-9fa8-408c2beba9ab None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:37.106912 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb05240-264d-4f8d-8a35-75195dfdb22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:37.107740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb05240-264d-4f8d-8a35-75195dfdb22d None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:37.108024 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb05240-264d-4f8d-8a35-75195dfdb22d None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:37.108297 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb05240-264d-4f8d-8a35-75195dfdb22d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:37.119198 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb05240-264d-4f8d-8a35-75195dfdb22d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:00:37.130767 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-3fb05240-264d-4f8d-8a35-75195dfdb22d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:00:37.130767 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:00:37.130767 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:00:37.130767 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:00:37.130767 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:00:37.130767 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:00:37.130767 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:00:37.130767 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:00:37.130767 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:00:37.130767 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:00:37.130767 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:00:37.130767 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:00:37.130767 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:00:37.130767 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:00:37.130767 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:00:37.130767 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:00:37.131816 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:00:37.131816 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:00:37.131816 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:00:37.131816 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:00:37.131816 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:00:37.131816 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:00:37.131816 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:00:37.131816 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:00:37.131816 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:00:37.131816 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:00:37.131816 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:00:37.131816 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:00:37.131816 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:00:37.131816 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:00:37.131816 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:00:37.131816 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:00:37.132656 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:00:37.132656 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:00:37.132656 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:00:37.132656 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:00:37.132656 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:00:37.132656 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:00:37.132656 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:00:37.132656 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:00:37.132656 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:00:37.132656 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:00:37.132656 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:00:37.132656 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:00:37.132656 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 7/14] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jul 13 23:00:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:00:37.138458 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb05240-264d-4f8d-8a35-75195dfdb22d None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:37.200346 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f668ab7f-51ee-4864-9fa8-408c2beba9ab None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:37.202615 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:37.218418 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:37.237171 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb05240-264d-4f8d-8a35-75195dfdb22d None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:37.238360 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4e312-53bc-4b9a-a3f9-042018f40a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:37.239253 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4e312-53bc-4b9a-a3f9-042018f40a53 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:37.239551 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4e312-53bc-4b9a-a3f9-042018f40a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:37.239846 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4e312-53bc-4b9a-a3f9-042018f40a53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:37.271611 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c7f191-dcb2-4654-a910-fb647ad26679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:37.272277 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c7f191-dcb2-4654-a910-fb647ad26679 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:37.272559 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c7f191-dcb2-4654-a910-fb647ad26679 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:37.272842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c7f191-dcb2-4654-a910-fb647ad26679 None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:37.273600 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c7f191-dcb2-4654-a910-fb647ad26679 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:00:37.273909 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c7f191-dcb2-4654-a910-fb647ad26679 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:37.275314 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c7f191-dcb2-4654-a910-fb647ad26679 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:37.287217 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 8/15] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:00:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:00:37.292549 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c7f191-dcb2-4654-a910-fb647ad26679 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:37.305260 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4e312-53bc-4b9a-a3f9-042018f40a53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:37.306801 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4e312-53bc-4b9a-a3f9-042018f40a53 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:37.392745 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c7f191-dcb2-4654-a910-fb647ad26679 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:37.407454 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 8/16] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:00:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 13 23:00:37.413071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4e312-53bc-4b9a-a3f9-042018f40a53 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:37.415767 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:37.425902 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e368a-6748-4355-b201-f62edfe6418e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:37.426780 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e368a-6748-4355-b201-f62edfe6418e None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:37.427070 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e368a-6748-4355-b201-f62edfe6418e None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:37.427348 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e368a-6748-4355-b201-f62edfe6418e None admin] PATH_INFO: `/v3/users/7f1cf8ea49bb4e7fb83c77a4a463d6fd` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:37.428591 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e368a-6748-4355-b201-f62edfe6418e None admin] RBAC: Authorizing `identity:get_user(user_id=7f1cf8ea49bb4e7fb83c77a4a463d6fd)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:37.430066 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e368a-6748-4355-b201-f62edfe6418e None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:37.431739 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 9/17] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:00:37 2022] GET /identity/v3/users/7f1cf8ea49bb4e7fb83c77a4a463d6fd => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:00:37.437549 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e368a-6748-4355-b201-f62edfe6418e None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:37.452107 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:37.510914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4e312-53bc-4b9a-a3f9-042018f40a53 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:37.537317 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e368a-6748-4355-b201-f62edfe6418e None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:37.544278 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92b02d-0b66-4db0-a8c6-83f62f18ae7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:37.545075 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92b02d-0b66-4db0-a8c6-83f62f18ae7f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:37.545362 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92b02d-0b66-4db0-a8c6-83f62f18ae7f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:37.545639 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92b02d-0b66-4db0-a8c6-83f62f18ae7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:37.564242 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db92b02d-0b66-4db0-a8c6-83f62f18ae7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:37.565727 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db92b02d-0b66-4db0-a8c6-83f62f18ae7f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:37.570721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eade7c4-5dc8-4845-9c8b-6d9e2fdccd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:37.571526 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eade7c4-5dc8-4845-9c8b-6d9e2fdccd68 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:37.571858 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eade7c4-5dc8-4845-9c8b-6d9e2fdccd68 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:37.572141 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eade7c4-5dc8-4845-9c8b-6d9e2fdccd68 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:37.573320 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eade7c4-5dc8-4845-9c8b-6d9e2fdccd68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:37.575028 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eade7c4-5dc8-4845-9c8b-6d9e2fdccd68 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:37.576677 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 10/18] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:00:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:00:37.582290 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eade7c4-5dc8-4845-9c8b-6d9e2fdccd68 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:37.588618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-db92b02d-0b66-4db0-a8c6-83f62f18ae7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:00:37.588618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:00:37.588618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:00:37.588618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:00:37.588618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:00:37.588618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:00:37.588618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:00:37.588618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:00:37.588618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:00:37.588618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:00:37.588618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:00:37.588618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:00:37.588618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:00:37.588618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:00:37.588618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:00:37.589491 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:00:37.589491 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:00:37.589491 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:00:37.589491 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:00:37.589491 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:00:37.589491 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:00:37.589491 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:00:37.589491 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:00:37.589491 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:00:37.589491 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:00:37.589491 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:00:37.589491 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:00:37.589491 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:00:37.589491 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:00:37.589491 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:00:37.589491 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:00:37.589491 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:00:37.590272 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:00:37.590272 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:00:37.590272 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:00:37.590272 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:00:37.590272 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 9/19] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:00:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:00:37.595802 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92b02d-0b66-4db0-a8c6-83f62f18ae7f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:37.650731 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:37.661160 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:37.680529 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eade7c4-5dc8-4845-9c8b-6d9e2fdccd68 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:37.695617 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92b02d-0b66-4db0-a8c6-83f62f18ae7f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:37.716814 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ddc36a-b1ff-4572-8164-1b50b6b20373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:37.717471 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ddc36a-b1ff-4572-8164-1b50b6b20373 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:37.717754 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ddc36a-b1ff-4572-8164-1b50b6b20373 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:37.718058 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ddc36a-b1ff-4572-8164-1b50b6b20373 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:37.718891 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ddc36a-b1ff-4572-8164-1b50b6b20373 None admin] RBAC: Adding query filter params (role.id=8cfd0b6e71e14e0983c3eb4d7478e027, scope.domain.id=default, user.id=7f1cf8ea49bb4e7fb83c77a4a463d6fd) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:00:37.719206 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ddc36a-b1ff-4572-8164-1b50b6b20373 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:37.720679 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ddc36a-b1ff-4572-8164-1b50b6b20373 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:37.727297 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 11/20] 10.222.0.60 () {56 vars in 1322 bytes} [Wed Jul 13 23:00:37 2022] GET /identity/v3/role_assignments?user.id=7f1cf8ea49bb4e7fb83c77a4a463d6fd&scope.domain.id=default&role.id=8cfd0b6e71e14e0983c3eb4d7478e027 => generated 233 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:00:37.733763 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19c6ca4-489f-4541-9ff1-ae01a9c075f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:37.734532 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c6ca4-489f-4541-9ff1-ae01a9c075f1 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:37.734882 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c6ca4-489f-4541-9ff1-ae01a9c075f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:37.735229 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c6ca4-489f-4541-9ff1-ae01a9c075f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:37.736206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19c6ca4-489f-4541-9ff1-ae01a9c075f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:00:37.736577 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19c6ca4-489f-4541-9ff1-ae01a9c075f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:37.738086 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19c6ca4-489f-4541-9ff1-ae01a9c075f1 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:37.758803 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 10/21] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:00:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:00:40.106645 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c85b63-17f8-4f94-9774-594d37fa7e8f None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:40.107768 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c85b63-17f8-4f94-9774-594d37fa7e8f None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:40.108275 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c85b63-17f8-4f94-9774-594d37fa7e8f None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:40.109943 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 12/22] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:00:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:00:40.115714 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e8e56-93fd-4705-9d2d-83f7ea25368b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:40.116602 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e8e56-93fd-4705-9d2d-83f7ea25368b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:40.117079 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e8e56-93fd-4705-9d2d-83f7ea25368b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:40.144639 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-a80e8e56-93fd-4705-9d2d-83f7ea25368b None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:00:40.146241 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a80e8e56-93fd-4705-9d2d-83f7ea25368b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:00:40.237904 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 11/23] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:00:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:00:40.272189 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c85b63-17f8-4f94-9774-594d37fa7e8f None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:40.420037 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c85b63-17f8-4f94-9774-594d37fa7e8f None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:40.469746 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021dae14-9a41-43c2-87e2-23e3f2f10a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:40.470564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021dae14-9a41-43c2-87e2-23e3f2f10a8a None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:40.470848 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021dae14-9a41-43c2-87e2-23e3f2f10a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:40.471123 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021dae14-9a41-43c2-87e2-23e3f2f10a8a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:40.487255 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7f4f3-4dd8-4d5c-9380-aeef15c9f522 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:40.487924 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7f4f3-4dd8-4d5c-9380-aeef15c9f522 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:40.488402 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7f4f3-4dd8-4d5c-9380-aeef15c9f522 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:40.488663 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021dae14-9a41-43c2-87e2-23e3f2f10a8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:00:40.490191 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 12/24] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:00:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:00:40.496893 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f9c27-9f9e-4c65-a763-8ea912e59fe4 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:40.497464 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f9c27-9f9e-4c65-a763-8ea912e59fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:40.497940 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f9c27-9f9e-4c65-a763-8ea912e59fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:40.502321 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-021dae14-9a41-43c2-87e2-23e3f2f10a8a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:00:40.502321 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:00:40.502321 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:00:40.502321 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:00:40.502321 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:00:40.502321 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:00:40.502321 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:00:40.502321 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:00:40.502321 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:00:40.502321 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:00:40.502321 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:00:40.502321 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:00:40.502321 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:00:40.502321 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:00:40.502321 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:00:40.502321 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:00:40.503158 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:00:40.503158 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:00:40.503158 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:00:40.503158 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:00:40.503158 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:00:40.503158 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:00:40.503158 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:00:40.503158 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:00:40.503158 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:00:40.503158 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:00:40.503158 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:00:40.503158 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:00:40.503158 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:00:40.503158 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:00:40.503158 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:00:40.503158 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:00:40.503914 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:00:40.503914 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:00:40.503914 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:00:40.503914 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:00:40.503914 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:00:40.503914 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:00:40.503914 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:00:40.503914 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:00:40.503914 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:00:40.503914 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:00:40.503914 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:00:40.503914 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:00:40.503914 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 13/25] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jul 13 23:00:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:00:40.509515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021dae14-9a41-43c2-87e2-23e3f2f10a8a None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:40.516760 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:40.542740 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-dd6f9c27-9f9e-4c65-a763-8ea912e59fe4 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:00:40.544219 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-dd6f9c27-9f9e-4c65-a763-8ea912e59fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:00:40.647889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:40.690551 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021dae14-9a41-43c2-87e2-23e3f2f10a8a None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:40.711073 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 13/26] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:00:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:00:40.747460 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fdaeab-b506-4fa4-a6f9-1a9ca1c3e3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:40.747818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f9c27-9f9e-4c65-a763-8ea912e59fe4 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:40.748224 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fdaeab-b506-4fa4-a6f9-1a9ca1c3e3b3 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:40.748508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fdaeab-b506-4fa4-a6f9-1a9ca1c3e3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:40.748788 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fdaeab-b506-4fa4-a6f9-1a9ca1c3e3b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:40.749567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fdaeab-b506-4fa4-a6f9-1a9ca1c3e3b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:00:40.749875 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fdaeab-b506-4fa4-a6f9-1a9ca1c3e3b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:40.751314 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fdaeab-b506-4fa4-a6f9-1a9ca1c3e3b3 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:40.764245 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 14/27] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:00:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:00:40.768029 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b09276-cf93-4ba1-9064-6c5a75005f09 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:40.768743 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b09276-cf93-4ba1-9064-6c5a75005f09 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:40.770590 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b09276-cf93-4ba1-9064-6c5a75005f09 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:40.773662 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 15/28] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:00:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:00:40.779556 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b09276-cf93-4ba1-9064-6c5a75005f09 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:40.851240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:40.891303 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f9c27-9f9e-4c65-a763-8ea912e59fe4 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:40.893001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b09276-cf93-4ba1-9064-6c5a75005f09 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:40.936231 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f8987f-2a4d-49f8-b2b5-b89e7ce5fc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:40.937234 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f8987f-2a4d-49f8-b2b5-b89e7ce5fc79 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:40.937432 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f8987f-2a4d-49f8-b2b5-b89e7ce5fc79 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:40.937678 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f8987f-2a4d-49f8-b2b5-b89e7ce5fc79 None admin] PATH_INFO: `/v3/users/7f1cf8ea49bb4e7fb83c77a4a463d6fd` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:40.938904 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f8987f-2a4d-49f8-b2b5-b89e7ce5fc79 None admin] RBAC: Authorizing `identity:get_user(user_id=7f1cf8ea49bb4e7fb83c77a4a463d6fd)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:40.940540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f8987f-2a4d-49f8-b2b5-b89e7ce5fc79 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:40.942454 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 16/29] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:00:40 2022] GET /identity/v3/users/7f1cf8ea49bb4e7fb83c77a4a463d6fd => generated 247 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:00:40.944357 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f128a0-79fb-401a-a98c-45c9e6e64763 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:40.944918 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f128a0-79fb-401a-a98c-45c9e6e64763 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:40.945397 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f128a0-79fb-401a-a98c-45c9e6e64763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:40.951673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e07e99-57fe-43ec-b463-3ec253dfbaa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:40.952986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e07e99-57fe-43ec-b463-3ec253dfbaa8 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:40.954471 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e07e99-57fe-43ec-b463-3ec253dfbaa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:40.954947 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e07e99-57fe-43ec-b463-3ec253dfbaa8 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:40.955974 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e07e99-57fe-43ec-b463-3ec253dfbaa8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:40.957595 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e07e99-57fe-43ec-b463-3ec253dfbaa8 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:40.965794 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-78f128a0-79fb-401a-a98c-45c9e6e64763 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:00:40.967120 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-78f128a0-79fb-401a-a98c-45c9e6e64763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:00:41.049330 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 17/30] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:00:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:00:41.056240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead21d2b-fe3d-4673-8748-e00959ee60de None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:41.056734 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead21d2b-fe3d-4673-8748-e00959ee60de None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:41.059273 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead21d2b-fe3d-4673-8748-e00959ee60de None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:41.060927 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 18/31] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:00:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:00:41.061935 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead21d2b-fe3d-4673-8748-e00959ee60de None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:41.100988 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 14/32] 10.222.0.60 () {60 vars in 1131 bytes} [Wed Jul 13 23:00:40 2022] POST /identity/v3/roles => generated 219 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:00:41.102707 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece73ef-0c86-450c-a65e-311573ab2c07 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:41.103073 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece73ef-0c86-450c-a65e-311573ab2c07 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:41.103420 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece73ef-0c86-450c-a65e-311573ab2c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:41.128272 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-dece73ef-0c86-450c-a65e-311573ab2c07 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:00:41.130269 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-dece73ef-0c86-450c-a65e-311573ab2c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:00:41.162731 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:41.187377 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:41.218630 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead21d2b-fe3d-4673-8748-e00959ee60de None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:41.269180 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf48478-b7e9-4428-b7ab-aea4e6a6d4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:41.270950 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf48478-b7e9-4428-b7ab-aea4e6a6d4be None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:41.272517 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf48478-b7e9-4428-b7ab-aea4e6a6d4be None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:41.273688 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf48478-b7e9-4428-b7ab-aea4e6a6d4be None admin] PATH_INFO: `/v3/domains/default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:41.275688 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf48478-b7e9-4428-b7ab-aea4e6a6d4be None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:41.278747 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 15/33] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:00:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:00:41.280775 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf48478-b7e9-4428-b7ab-aea4e6a6d4be None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:41.280884 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dece73ef-0c86-450c-a65e-311573ab2c07 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:41.283369 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 19/34] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:00:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:00:41.289826 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf48478-b7e9-4428-b7ab-aea4e6a6d4be None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:41.455807 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dece73ef-0c86-450c-a65e-311573ab2c07 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:41.478419 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf48478-b7e9-4428-b7ab-aea4e6a6d4be None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:41.502078 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd47365-4a7f-412a-bbb7-905776c53693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:41.502762 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd47365-4a7f-412a-bbb7-905776c53693 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:41.503044 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd47365-4a7f-412a-bbb7-905776c53693 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:41.503321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd47365-4a7f-412a-bbb7-905776c53693 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:41.504128 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd47365-4a7f-412a-bbb7-905776c53693 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:41.506563 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd47365-4a7f-412a-bbb7-905776c53693 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:41.520591 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:41.525948 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 16/35] 10.222.0.60 () {60 vars in 1131 bytes} [Wed Jul 13 23:00:41 2022] POST /identity/v3/roles => generated 225 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:00:41.527777 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e33a932-fa53-4c55-b22c-158bb22fa350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:41.527926 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd47365-4a7f-412a-bbb7-905776c53693 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:41.529831 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33a932-fa53-4c55-b22c-158bb22fa350 None admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:41.531324 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33a932-fa53-4c55-b22c-158bb22fa350 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:41.532733 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e33a932-fa53-4c55-b22c-158bb22fa350 None admin] PATH_INFO: `/v3/domains/default/users/7f1cf8ea49bb4e7fb83c77a4a463d6fd/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:41.535273 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e33a932-fa53-4c55-b22c-158bb22fa350 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=7f1cf8ea49bb4e7fb83c77a4a463d6fd, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:41.537457 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e33a932-fa53-4c55-b22c-158bb22fa350 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:41.566496 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:41.653483 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd47365-4a7f-412a-bbb7-905776c53693 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:41.678317 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 20/36] 10.222.0.60 () {58 vars in 1361 bytes} [Wed Jul 13 23:00:41 2022] PUT /identity/v3/domains/default/users/7f1cf8ea49bb4e7fb83c77a4a463d6fd/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:00:41.695467 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b2eb5-2b33-4c3d-97b5-7327b1c3103c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:41.696383 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b2eb5-2b33-4c3d-97b5-7327b1c3103c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:41.696671 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b2eb5-2b33-4c3d-97b5-7327b1c3103c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:41.696947 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b2eb5-2b33-4c3d-97b5-7327b1c3103c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:41.711964 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b2eb5-2b33-4c3d-97b5-7327b1c3103c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:41.713419 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b2eb5-2b33-4c3d-97b5-7327b1c3103c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:41.734126 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-e53b2eb5-2b33-4c3d-97b5-7327b1c3103c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:00:41.734126 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:00:41.734126 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:00:41.734126 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:00:41.734126 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:00:41.734126 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:00:41.734126 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:00:41.734126 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:00:41.734126 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:00:41.734126 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:00:41.734126 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:00:41.734126 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:00:41.734126 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:00:41.734126 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:00:41.734126 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:00:41.735016 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:00:41.735016 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:00:41.735016 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:00:41.735016 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:00:41.735016 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:00:41.735016 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:00:41.735016 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:00:41.735016 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:00:41.735016 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:00:41.735016 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:00:41.735016 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:00:41.735016 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:00:41.735016 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:00:41.735016 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:00:41.735016 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:00:41.735016 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:00:41.735016 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:00:41.735815 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:00:41.735815 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:00:41.735815 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:00:41.735815 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:00:41.735815 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 17/37] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:00:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:00:41.742349 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e33a932-fa53-4c55-b22c-158bb22fa350 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:41.798717 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae12e30-1b62-4ca5-9110-ae6a20a054d8 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:41.799361 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae12e30-1b62-4ca5-9110-ae6a20a054d8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:41.799880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae12e30-1b62-4ca5-9110-ae6a20a054d8 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:41.801792 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 18/38] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:00:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:00:41.808929 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f0f228-1588-4a07-a6ad-daf5a4ee40fb None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:41.809457 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f0f228-1588-4a07-a6ad-daf5a4ee40fb None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:41.809960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f0f228-1588-4a07-a6ad-daf5a4ee40fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:41.829150 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-59f0f228-1588-4a07-a6ad-daf5a4ee40fb None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:00:41.830764 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-59f0f228-1588-4a07-a6ad-daf5a4ee40fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:00:41.908858 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e33a932-fa53-4c55-b22c-158bb22fa350 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:41.919862 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 19/39] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:00:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 23:00:41.948644 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f0f228-1588-4a07-a6ad-daf5a4ee40fb None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:41.955207 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f971a7-4649-4b7f-8825-43edf0bf8418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:41.955986 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f971a7-4649-4b7f-8825-43edf0bf8418 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:41.956275 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f971a7-4649-4b7f-8825-43edf0bf8418 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:41.956557 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f971a7-4649-4b7f-8825-43edf0bf8418 None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:41.957313 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f971a7-4649-4b7f-8825-43edf0bf8418 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:00:41.957623 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f971a7-4649-4b7f-8825-43edf0bf8418 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:41.959926 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f971a7-4649-4b7f-8825-43edf0bf8418 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:41.966319 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:41.984988 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 21/40] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:00:41.992063 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f971a7-4649-4b7f-8825-43edf0bf8418 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:42.058210 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:42.084066 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f0f228-1588-4a07-a6ad-daf5a4ee40fb None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:42.130196 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed810b1-ceb6-4618-a050-64d860a91460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:42.130979 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed810b1-ceb6-4618-a050-64d860a91460 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:42.131286 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed810b1-ceb6-4618-a050-64d860a91460 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:42.131671 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed810b1-ceb6-4618-a050-64d860a91460 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:42.132546 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed810b1-ceb6-4618-a050-64d860a91460 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:42.134001 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed810b1-ceb6-4618-a050-64d860a91460 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:42.154923 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 20/41] 10.222.0.60 () {60 vars in 1131 bytes} [Wed Jul 13 23:00:41 2022] POST /identity/v3/roles => generated 223 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:00:42.156838 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f454890b-ebca-41e2-98a9-63ecdff835e8 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:42.157387 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f454890b-ebca-41e2-98a9-63ecdff835e8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:42.157926 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f454890b-ebca-41e2-98a9-63ecdff835e8 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:42.159791 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 21/42] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:00:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:00:42.174171 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8883fc-013f-4dc3-870a-0dffa06c7dbb None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:42.174896 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8883fc-013f-4dc3-870a-0dffa06c7dbb None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:42.175415 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8883fc-013f-4dc3-870a-0dffa06c7dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:42.197089 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-7b8883fc-013f-4dc3-870a-0dffa06c7dbb None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:00:42.200409 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7b8883fc-013f-4dc3-870a-0dffa06c7dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:00:42.235465 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f971a7-4649-4b7f-8825-43edf0bf8418 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:42.290090 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd9bb2-b606-4f47-88ae-389ae7d5858d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:42.290995 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd9bb2-b606-4f47-88ae-389ae7d5858d None admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:42.291566 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd9bb2-b606-4f47-88ae-389ae7d5858d None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:42.292085 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd9bb2-b606-4f47-88ae-389ae7d5858d None admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:42.293147 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdd9bb2-b606-4f47-88ae-389ae7d5858d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:42.293523 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:42.294881 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdd9bb2-b606-4f47-88ae-389ae7d5858d None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:42.300436 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 22/43] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:00:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 23:00:42.329615 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8883fc-013f-4dc3-870a-0dffa06c7dbb None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:42.342769 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 22/44] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Jul 13 23:00:41 2022] POST /identity/v3/projects => generated 301 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:00:42.468648 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8883fc-013f-4dc3-870a-0dffa06c7dbb None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:42.511099 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21afcf68-8bcc-4df0-b1e8-79f3ee6aa19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:42.512373 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21afcf68-8bcc-4df0-b1e8-79f3ee6aa19b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:42.512660 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21afcf68-8bcc-4df0-b1e8-79f3ee6aa19b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:42.512947 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21afcf68-8bcc-4df0-b1e8-79f3ee6aa19b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:42.514424 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21afcf68-8bcc-4df0-b1e8-79f3ee6aa19b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:42.515926 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21afcf68-8bcc-4df0-b1e8-79f3ee6aa19b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:42.517610 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 23/45] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:00:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:00:42.526046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd9bb2-b606-4f47-88ae-389ae7d5858d None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:42.646121 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd9bb2-b606-4f47-88ae-389ae7d5858d None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:42.648453 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:42.682735 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff18ab-5d89-4882-8f3c-e313695cf2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:42.683928 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff18ab-5d89-4882-8f3c-e313695cf2f2 None admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:42.684217 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff18ab-5d89-4882-8f3c-e313695cf2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:42.684498 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff18ab-5d89-4882-8f3c-e313695cf2f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:42.685224 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cff18ab-5d89-4882-8f3c-e313695cf2f2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:42.686699 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cff18ab-5d89-4882-8f3c-e313695cf2f2 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:42.711596 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 23/46] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Jul 13 23:00:42 2022] POST /identity/v3/projects => generated 312 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:00:43.955505 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ae4fa-94aa-428d-9d9e-180215eca156 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:43.956049 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ae4fa-94aa-428d-9d9e-180215eca156 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:43.956142 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ae4fa-94aa-428d-9d9e-180215eca156 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:43.957484 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 24/47] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:00:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:00:43.963107 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a6a31-6ea4-43b1-914c-7735060972d2 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:43.963618 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a6a31-6ea4-43b1-914c-7735060972d2 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:43.963864 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a6a31-6ea4-43b1-914c-7735060972d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:43.981123 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-a77a6a31-6ea4-43b1-914c-7735060972d2 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:00:43.982647 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-a77a6a31-6ea4-43b1-914c-7735060972d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:00:44.049109 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 24/48] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:00:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 23:00:44.073231 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ae4fa-94aa-428d-9d9e-180215eca156 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:44.142492 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:44.177808 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ae4fa-94aa-428d-9d9e-180215eca156 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:44.211919 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2241a63-7a6e-447c-8d7d-218723fff9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:44.212634 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2241a63-7a6e-447c-8d7d-218723fff9ea None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:44.212922 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2241a63-7a6e-447c-8d7d-218723fff9ea None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:44.213206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2241a63-7a6e-447c-8d7d-218723fff9ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:44.224726 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2241a63-7a6e-447c-8d7d-218723fff9ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:00:44.234063 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-e2241a63-7a6e-447c-8d7d-218723fff9ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:00:44.234063 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:00:44.234063 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:00:44.234063 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:00:44.234063 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:00:44.234063 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:00:44.234063 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:00:44.234063 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:00:44.234063 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:00:44.234063 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:00:44.234063 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:00:44.234063 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:00:44.234063 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:00:44.234063 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:00:44.234063 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:00:44.234063 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:00:44.234889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:00:44.234889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:00:44.234889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:00:44.234889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:00:44.234889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:00:44.234889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:00:44.234889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:00:44.234889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:00:44.234889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:00:44.234889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:00:44.234889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:00:44.234889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:00:44.234889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:00:44.234889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:00:44.234889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:00:44.234889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:00:44.235731 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:00:44.235731 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:00:44.235731 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:00:44.235731 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:00:44.235731 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:00:44.235731 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:00:44.235731 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:00:44.235731 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:00:44.235731 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:00:44.235731 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:00:44.235731 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:00:44.235731 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:00:44.239936 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 25/49] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jul 13 23:00:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:00:44.241502 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77a6a31-6ea4-43b1-914c-7735060972d2 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:44.323807 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:44.339919 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77a6a31-6ea4-43b1-914c-7735060972d2 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:44.376362 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953fddaa-5631-4a10-aab3-76a86c94ab4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:44.377037 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953fddaa-5631-4a10-aab3-76a86c94ab4d None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:44.377320 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953fddaa-5631-4a10-aab3-76a86c94ab4d None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:44.377597 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953fddaa-5631-4a10-aab3-76a86c94ab4d None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:44.378347 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953fddaa-5631-4a10-aab3-76a86c94ab4d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:00:44.378656 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953fddaa-5631-4a10-aab3-76a86c94ab4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:44.380089 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953fddaa-5631-4a10-aab3-76a86c94ab4d None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:44.388466 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 25/50] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:00:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:00:44.394008 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2241a63-7a6e-447c-8d7d-218723fff9ea None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:44.494133 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2241a63-7a6e-447c-8d7d-218723fff9ea None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:44.520200 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:44.529375 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee9f14d-0d2b-43fb-b10c-355e0e155e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:44.530264 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9f14d-0d2b-43fb-b10c-355e0e155e19 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:44.530550 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9f14d-0d2b-43fb-b10c-355e0e155e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:44.530827 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee9f14d-0d2b-43fb-b10c-355e0e155e19 None admin] PATH_INFO: `/v3/users/7f1cf8ea49bb4e7fb83c77a4a463d6fd` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:44.532358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee9f14d-0d2b-43fb-b10c-355e0e155e19 None admin] RBAC: Authorizing `identity:get_user(user_id=7f1cf8ea49bb4e7fb83c77a4a463d6fd)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:44.533847 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee9f14d-0d2b-43fb-b10c-355e0e155e19 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:44.535563 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 26/51] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:00:44 2022] GET /identity/v3/users/7f1cf8ea49bb4e7fb83c77a4a463d6fd => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:00:44.541003 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953fddaa-5631-4a10-aab3-76a86c94ab4d None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:44.642111 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953fddaa-5631-4a10-aab3-76a86c94ab4d None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:44.676676 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99869a4-aa99-4974-bd0e-7ffefc534270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:44.677471 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99869a4-aa99-4974-bd0e-7ffefc534270 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:44.677748 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99869a4-aa99-4974-bd0e-7ffefc534270 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:44.678018 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99869a4-aa99-4974-bd0e-7ffefc534270 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:44.679192 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99869a4-aa99-4974-bd0e-7ffefc534270 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:44.680689 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99869a4-aa99-4974-bd0e-7ffefc534270 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:44.682330 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 26/52] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:00:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:00:44.688069 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee9f14d-0d2b-43fb-b10c-355e0e155e19 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:44.784991 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041e102-4a82-40cd-9924-190fc956f06c None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:44.785168 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041e102-4a82-40cd-9924-190fc956f06c None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:44.785457 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041e102-4a82-40cd-9924-190fc956f06c None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:44.786913 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 27/53] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:00:44.791456 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee9f14d-0d2b-43fb-b10c-355e0e155e19 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:44.792772 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991683ee-0c2b-41d6-9c5d-6b81f141a7ee None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:44.793065 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991683ee-0c2b-41d6-9c5d-6b81f141a7ee None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:44.793835 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991683ee-0c2b-41d6-9c5d-6b81f141a7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:44.811444 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-991683ee-0c2b-41d6-9c5d-6b81f141a7ee None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:00:44.812727 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-991683ee-0c2b-41d6-9c5d-6b81f141a7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:00:44.824961 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:44.827800 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b38e4-8c71-4ea6-88f4-d8c90b94cd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:44.828519 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b38e4-8c71-4ea6-88f4-d8c90b94cd47 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:44.828822 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b38e4-8c71-4ea6-88f4-d8c90b94cd47 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:44.829084 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828b38e4-8c71-4ea6-88f4-d8c90b94cd47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:44.829940 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828b38e4-8c71-4ea6-88f4-d8c90b94cd47 None admin] RBAC: Adding query filter params (role.id=8cfd0b6e71e14e0983c3eb4d7478e027, scope.domain.id=default, user.id=7f1cf8ea49bb4e7fb83c77a4a463d6fd) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:00:44.830254 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828b38e4-8c71-4ea6-88f4-d8c90b94cd47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:44.831832 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828b38e4-8c71-4ea6-88f4-d8c90b94cd47 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:44.843304 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 27/54] 10.222.0.60 () {56 vars in 1322 bytes} [Wed Jul 13 23:00:44 2022] GET /identity/v3/role_assignments?user.id=7f1cf8ea49bb4e7fb83c77a4a463d6fd&scope.domain.id=default&role.id=8cfd0b6e71e14e0983c3eb4d7478e027 => generated 530 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:00:44.893119 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 28/55] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:00:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 23:00:44.917865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b38e4-8c71-4ea6-88f4-d8c90b94cd47 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:45.088871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:45.130499 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828b38e4-8c71-4ea6-88f4-d8c90b94cd47 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:45.163647 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4013d4d3-3d00-4961-9ee0-7503ee5a0f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:45.164447 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013d4d3-3d00-4961-9ee0-7503ee5a0f7d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:45.164736 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013d4d3-3d00-4961-9ee0-7503ee5a0f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:45.165020 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013d4d3-3d00-4961-9ee0-7503ee5a0f7d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:45.166200 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4013d4d3-3d00-4961-9ee0-7503ee5a0f7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:45.167710 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4013d4d3-3d00-4961-9ee0-7503ee5a0f7d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:45.169354 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 28/56] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:00:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:00:45.175237 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991683ee-0c2b-41d6-9c5d-6b81f141a7ee None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:45.272000 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991683ee-0c2b-41d6-9c5d-6b81f141a7ee None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:45.304392 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb87e5-aeac-4cb9-8cdc-e1023a9acda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:45.305050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb87e5-aeac-4cb9-8cdc-e1023a9acda6 None admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:45.305335 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb87e5-aeac-4cb9-8cdc-e1023a9acda6 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:45.305615 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb87e5-aeac-4cb9-8cdc-e1023a9acda6 None admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:45.306323 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeb87e5-aeac-4cb9-8cdc-e1023a9acda6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:45.307900 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeb87e5-aeac-4cb9-8cdc-e1023a9acda6 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:45.326473 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:45.333634 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 29/57] 10.222.0.60 () {60 vars in 1140 bytes} [Wed Jul 13 23:00:45 2022] POST /identity/v3/projects => generated 298 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:00:47.383012 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e5aac-9148-46eb-9bc2-b6638d7b132a None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:47.383507 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e5aac-9148-46eb-9bc2-b6638d7b132a None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:47.383650 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e5aac-9148-46eb-9bc2-b6638d7b132a None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:47.384981 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 29/58] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:00:47.390283 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1f36e-9cab-46a7-821b-f9b4e5b478c6 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:47.390687 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1f36e-9cab-46a7-821b-f9b4e5b478c6 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:47.390919 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1f36e-9cab-46a7-821b-f9b4e5b478c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:47.407193 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-6fd1f36e-9cab-46a7-821b-f9b4e5b478c6 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:00:47.408417 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-6fd1f36e-9cab-46a7-821b-f9b4e5b478c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:00:47.472259 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 30/59] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:00:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:00:47.498172 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e5aac-9148-46eb-9bc2-b6638d7b132a None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:47.605824 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e5aac-9148-46eb-9bc2-b6638d7b132a None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:47.630285 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:47.638815 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374025eb-24d4-4f94-b5fe-0c6f5bffd990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:47.639634 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374025eb-24d4-4f94-b5fe-0c6f5bffd990 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:47.639923 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374025eb-24d4-4f94-b5fe-0c6f5bffd990 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:47.640202 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374025eb-24d4-4f94-b5fe-0c6f5bffd990 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:47.641386 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374025eb-24d4-4f94-b5fe-0c6f5bffd990 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:47.642838 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374025eb-24d4-4f94-b5fe-0c6f5bffd990 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:47.644498 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 30/60] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:00:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:00:47.650177 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1f36e-9cab-46a7-821b-f9b4e5b478c6 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:47.745914 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1f36e-9cab-46a7-821b-f9b4e5b478c6 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:47.771108 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:47.779709 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae3c4ae-1cae-47c9-87f1-ca2f9f8e2602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:47.780377 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae3c4ae-1cae-47c9-87f1-ca2f9f8e2602 None admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:47.780664 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae3c4ae-1cae-47c9-87f1-ca2f9f8e2602 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:47.780939 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae3c4ae-1cae-47c9-87f1-ca2f9f8e2602 None admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:47.781657 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae3c4ae-1cae-47c9-87f1-ca2f9f8e2602 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:47.783128 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae3c4ae-1cae-47c9-87f1-ca2f9f8e2602 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:47.808923 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 31/61] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Jul 13 23:00:47 2022] POST /identity/v3/users => generated 275 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:00:50.942651 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f86d75-8cd0-4b86-b297-58c7775a7706 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:50.943568 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f86d75-8cd0-4b86-b297-58c7775a7706 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:50.944004 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f86d75-8cd0-4b86-b297-58c7775a7706 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:50.945549 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 31/62] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:00:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:00:50.951437 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee28bd9-6641-4d28-8251-fb40ff7fc005 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:50.952252 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee28bd9-6641-4d28-8251-fb40ff7fc005 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:50.952701 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee28bd9-6641-4d28-8251-fb40ff7fc005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:50.976837 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-0ee28bd9-6641-4d28-8251-fb40ff7fc005 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:00:50.978378 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-0ee28bd9-6641-4d28-8251-fb40ff7fc005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:00:51.046142 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c171ac1-8c1c-4fba-acee-9454dee688b0 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:51.046661 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c171ac1-8c1c-4fba-acee-9454dee688b0 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:51.047110 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c171ac1-8c1c-4fba-acee-9454dee688b0 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:51.048866 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 32/63] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:00:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:00:51.054942 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384f674-6c31-40e7-9fdd-e2984d6a755d None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:51.055574 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384f674-6c31-40e7-9fdd-e2984d6a755d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:51.056061 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384f674-6c31-40e7-9fdd-e2984d6a755d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:51.080013 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 32/64] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:00:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:00:51.091820 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-4384f674-6c31-40e7-9fdd-e2984d6a755d None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:00:51.093394 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4384f674-6c31-40e7-9fdd-e2984d6a755d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:00:51.103465 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee28bd9-6641-4d28-8251-fb40ff7fc005 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:51.207730 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 33/65] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:00:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 23:00:51.227820 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384f674-6c31-40e7-9fdd-e2984d6a755d None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:51.306140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee28bd9-6641-4d28-8251-fb40ff7fc005 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:51.343424 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:51.379293 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544dd9cd-a277-4714-ae70-0616e480cb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:51.380406 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544dd9cd-a277-4714-ae70-0616e480cb43 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:51.380838 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544dd9cd-a277-4714-ae70-0616e480cb43 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:51.381256 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544dd9cd-a277-4714-ae70-0616e480cb43 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:51.397348 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544dd9cd-a277-4714-ae70-0616e480cb43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:00:51.405415 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384f674-6c31-40e7-9fdd-e2984d6a755d None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:51.411581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-544dd9cd-a277-4714-ae70-0616e480cb43 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:00:51.411581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:00:51.411581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:00:51.411581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:00:51.411581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:00:51.411581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:00:51.411581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:00:51.411581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:00:51.411581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:00:51.411581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:00:51.411581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:00:51.411581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:00:51.411581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:00:51.411581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:00:51.411581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:00:51.411581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:00:51.412667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:00:51.412667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:00:51.412667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:00:51.412667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:00:51.412667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:00:51.412667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:00:51.412667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:00:51.412667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:00:51.412667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:00:51.412667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:00:51.412667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:00:51.412667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:00:51.412667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:00:51.412667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:00:51.412667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:00:51.412667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:00:51.413683 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:00:51.413683 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:00:51.413683 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:00:51.413683 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:00:51.413683 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:00:51.413683 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:00:51.413683 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:00:51.413683 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:00:51.413683 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:00:51.413683 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:00:51.413683 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:00:51.413683 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:00:51.423377 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 33/66] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jul 13 23:00:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:00:51.432678 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544dd9cd-a277-4714-ae70-0616e480cb43 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:51.464986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:51.489657 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6fbba-8987-4fb8-97b2-81a31ea15a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:51.491340 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6fbba-8987-4fb8-97b2-81a31ea15a9d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:51.491903 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6fbba-8987-4fb8-97b2-81a31ea15a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:51.492406 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6fbba-8987-4fb8-97b2-81a31ea15a9d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:51.512623 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6fbba-8987-4fb8-97b2-81a31ea15a9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:00:51.529299 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-82c6fbba-8987-4fb8-97b2-81a31ea15a9d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 23:00:51.529299 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:00:51.529299 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:00:51.529299 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:00:51.529299 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:00:51.529299 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:00:51.529299 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:00:51.529299 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:00:51.529299 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:00:51.529299 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:00:51.529299 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:00:51.529299 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:00:51.529299 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:00:51.529299 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:00:51.529299 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:00:51.529299 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:00:51.530205 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:00:51.530205 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:00:51.530205 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:00:51.530205 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:00:51.530205 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:00:51.530205 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:00:51.530205 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:00:51.530205 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:00:51.530205 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:00:51.530205 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:00:51.530205 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:00:51.530205 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:00:51.530205 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:00:51.530205 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:00:51.530205 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:00:51.530205 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:00:51.530940 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:00:51.530940 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:00:51.530940 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:00:51.530940 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:00:51.530940 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:00:51.530940 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:00:51.530940 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:00:51.530940 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:00:51.530940 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:00:51.530940 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:00:51.530940 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 23:00:51.530940 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:00:51.532177 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 34/67] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:00:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:00:51.534168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0417cea5-aaee-44cb-874d-9aa23d45b6d3 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:51.534692 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0417cea5-aaee-44cb-874d-9aa23d45b6d3 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:51.535182 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0417cea5-aaee-44cb-874d-9aa23d45b6d3 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:51.537889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 35/68] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:00:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:00:51.540123 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a0b64-0a4d-4d08-bae2-78303b7c4313 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:51.540604 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a0b64-0a4d-4d08-bae2-78303b7c4313 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:51.541105 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a0b64-0a4d-4d08-bae2-78303b7c4313 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:51.544517 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 36/69] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:00:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:00:51.548739 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a0b64-0a4d-4d08-bae2-78303b7c4313 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:51.584914 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544dd9cd-a277-4714-ae70-0616e480cb43 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:51.627975 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a07bad5-7ab8-4669-9d3f-2e498b97423b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:51.629084 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07bad5-7ab8-4669-9d3f-2e498b97423b None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:51.629563 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07bad5-7ab8-4669-9d3f-2e498b97423b None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:51.630049 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07bad5-7ab8-4669-9d3f-2e498b97423b None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:51.630935 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a07bad5-7ab8-4669-9d3f-2e498b97423b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:00:51.631243 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a07bad5-7ab8-4669-9d3f-2e498b97423b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:51.632726 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a07bad5-7ab8-4669-9d3f-2e498b97423b None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:51.647584 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 34/70] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:00:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:00:51.647584 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28439b7-7153-48ff-83f1-707e52103f9c None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:51.647584 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28439b7-7153-48ff-83f1-707e52103f9c None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:51.647584 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28439b7-7153-48ff-83f1-707e52103f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:51.667941 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-d28439b7-7153-48ff-83f1-707e52103f9c None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:00:51.669179 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-d28439b7-7153-48ff-83f1-707e52103f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:00:51.705752 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:51.712794 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a0b64-0a4d-4d08-bae2-78303b7c4313 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:51.772110 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 35/71] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:00:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:00:51.773985 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7547d-e842-4a91-9b2f-4e44e88b9b61 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:51.774429 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7547d-e842-4a91-9b2f-4e44e88b9b61 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:51.774887 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7547d-e842-4a91-9b2f-4e44e88b9b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:51.779381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd942f-a6a2-4e54-8862-ab879e8f4ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:51.780099 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd942f-a6a2-4e54-8862-ab879e8f4ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:51.780376 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd942f-a6a2-4e54-8862-ab879e8f4ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:51.780658 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd942f-a6a2-4e54-8862-ab879e8f4ec6 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:51.781421 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd942f-a6a2-4e54-8862-ab879e8f4ec6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:00:51.781723 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd942f-a6a2-4e54-8862-ab879e8f4ec6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:51.783159 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd942f-a6a2-4e54-8862-ab879e8f4ec6 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:51.797069 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 37/72] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jul 13 23:00:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:00:51.797948 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd942f-a6a2-4e54-8862-ab879e8f4ec6 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:51.803072 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-3cb7547d-e842-4a91-9b2f-4e44e88b9b61 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:00:51.804981 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-3cb7547d-e842-4a91-9b2f-4e44e88b9b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:00:51.853811 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:51.884215 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 36/73] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:00:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:00:51.886112 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd513e7-d89d-43d7-8ec2-2a960bfdd59b None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:51.886496 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd513e7-d89d-43d7-8ec2-2a960bfdd59b None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:51.886783 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd513e7-d89d-43d7-8ec2-2a960bfdd59b None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:51.888192 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 37/74] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:00:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:00:51.892328 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd513e7-d89d-43d7-8ec2-2a960bfdd59b None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:51.925449 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd942f-a6a2-4e54-8862-ab879e8f4ec6 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:51.967301 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db09e3-0f8c-4b59-a3de-245792ac67c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:51.968397 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db09e3-0f8c-4b59-a3de-245792ac67c8 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:51.968550 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db09e3-0f8c-4b59-a3de-245792ac67c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:51.968834 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db09e3-0f8c-4b59-a3de-245792ac67c8 None admin] PATH_INFO: `/v3/users/7f1cf8ea49bb4e7fb83c77a4a463d6fd` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:51.970121 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71db09e3-0f8c-4b59-a3de-245792ac67c8 None admin] RBAC: Authorizing `identity:get_user(user_id=7f1cf8ea49bb4e7fb83c77a4a463d6fd)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:51.971633 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71db09e3-0f8c-4b59-a3de-245792ac67c8 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:51.973326 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 38/75] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:00:51 2022] GET /identity/v3/users/7f1cf8ea49bb4e7fb83c77a4a463d6fd => generated 247 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:00:51.974102 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db09e3-0f8c-4b59-a3de-245792ac67c8 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:52.006678 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:52.017876 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd513e7-d89d-43d7-8ec2-2a960bfdd59b None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:52.074346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7234c08-827d-40c4-bd8d-59de2e06d5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:52.075125 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7234c08-827d-40c4-bd8d-59de2e06d5a7 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:52.075414 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7234c08-827d-40c4-bd8d-59de2e06d5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:52.075735 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7234c08-827d-40c4-bd8d-59de2e06d5a7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:52.092177 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7234c08-827d-40c4-bd8d-59de2e06d5a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:00:52.101346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-d7234c08-827d-40c4-bd8d-59de2e06d5a7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 23:00:52.101346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:00:52.101346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:00:52.101346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:00:52.101346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:00:52.101346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:00:52.101346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:00:52.101346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:00:52.101346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:00:52.101346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:00:52.101346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:00:52.101346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:00:52.101346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:00:52.101346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:00:52.101346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:00:52.101346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:00:52.103140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:00:52.103140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:00:52.103140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:00:52.103140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:00:52.103140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:00:52.103140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:00:52.103140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:00:52.103140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:00:52.103140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:00:52.103140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:00:52.103140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:00:52.103140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:00:52.103140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:00:52.103140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:00:52.103140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:00:52.103140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:00:52.104758 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:00:52.104758 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:00:52.104758 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:00:52.104758 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:00:52.104758 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:00:52.104758 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:00:52.104758 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:00:52.104758 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:00:52.104758 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:00:52.104758 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:00:52.104758 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 23:00:52.104758 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:00:52.104758 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 38/76] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jul 13 23:00:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:00:52.106127 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48887c16-db17-4bc8-9f19-dc40a0af75f5 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:52.106632 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48887c16-db17-4bc8-9f19-dc40a0af75f5 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:52.107103 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48887c16-db17-4bc8-9f19-dc40a0af75f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:52.112879 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db09e3-0f8c-4b59-a3de-245792ac67c8 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:52.127086 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-48887c16-db17-4bc8-9f19-dc40a0af75f5 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:00:52.128542 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-48887c16-db17-4bc8-9f19-dc40a0af75f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:00:52.131097 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:52.161607 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea73ab0-40c6-400e-a77b-8ef581c7d73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:52.163013 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea73ab0-40c6-400e-a77b-8ef581c7d73b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:52.163467 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea73ab0-40c6-400e-a77b-8ef581c7d73b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:52.163959 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea73ab0-40c6-400e-a77b-8ef581c7d73b None admin] PATH_INFO: `/v3/users/9159a16d47ec45bd861706097b1a5a73` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:52.189906 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea73ab0-40c6-400e-a77b-8ef581c7d73b None admin] RBAC: Authorizing `identity:get_user(user_id=9159a16d47ec45bd861706097b1a5a73)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:52.191557 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea73ab0-40c6-400e-a77b-8ef581c7d73b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:52.200098 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 39/77] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:00:51 2022] GET /identity/v3/users/9159a16d47ec45bd861706097b1a5a73 => generated 275 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:00:52.200098 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea73ab0-40c6-400e-a77b-8ef581c7d73b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:52.235651 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 39/78] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:00:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:00:52.236763 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48887c16-db17-4bc8-9f19-dc40a0af75f5 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:52.357630 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:52.365068 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea73ab0-40c6-400e-a77b-8ef581c7d73b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:52.382271 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48887c16-db17-4bc8-9f19-dc40a0af75f5 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:52.416083 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8031564-d21b-4ec4-8828-60eceebc8412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:52.417438 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8031564-d21b-4ec4-8828-60eceebc8412 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:52.417894 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8031564-d21b-4ec4-8828-60eceebc8412 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:52.418348 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8031564-d21b-4ec4-8828-60eceebc8412 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:52.437269 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8031564-d21b-4ec4-8828-60eceebc8412 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:00:52.438627 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae9672-a6eb-45b7-ad3e-f29462ef2a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:52.439719 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae9672-a6eb-45b7-ad3e-f29462ef2a22 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:52.440193 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae9672-a6eb-45b7-ad3e-f29462ef2a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:52.440654 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae9672-a6eb-45b7-ad3e-f29462ef2a22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:52.451783 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-d8031564-d21b-4ec4-8828-60eceebc8412 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 23:00:52.451783 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:00:52.451783 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:00:52.451783 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:00:52.451783 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:00:52.451783 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:00:52.451783 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:00:52.451783 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:00:52.451783 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:00:52.451783 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:00:52.451783 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:00:52.451783 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:00:52.451783 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:00:52.451783 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:00:52.451783 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:00:52.451783 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:00:52.453499 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:00:52.453499 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:00:52.453499 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:00:52.453499 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:00:52.453499 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:00:52.453499 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:00:52.453499 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:00:52.453499 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:00:52.453499 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:00:52.453499 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:00:52.453499 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:00:52.453499 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:00:52.453499 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:00:52.453499 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:00:52.453499 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:00:52.453499 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:00:52.454963 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:00:52.454963 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:00:52.454963 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:00:52.454963 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:00:52.454963 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:00:52.454963 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:00:52.454963 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:00:52.454963 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:00:52.454963 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:00:52.454963 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:00:52.454963 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 23:00:52.454963 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:00:52.454963 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 40/79] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:00:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:00:52.456625 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8031564-d21b-4ec4-8828-60eceebc8412 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:52.500409 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:52.540579 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ae9672-a6eb-45b7-ad3e-f29462ef2a22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:52.542674 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ae9672-a6eb-45b7-ad3e-f29462ef2a22 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:52.587298 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8031564-d21b-4ec4-8828-60eceebc8412 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:52.629719 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4b9f8a-9120-44af-ba1c-5028564d5109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:52.630548 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b9f8a-9120-44af-ba1c-5028564d5109 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:52.631015 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b9f8a-9120-44af-ba1c-5028564d5109 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:52.631386 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b9f8a-9120-44af-ba1c-5028564d5109 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:52.632552 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4b9f8a-9120-44af-ba1c-5028564d5109 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:00:52.633032 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4b9f8a-9120-44af-ba1c-5028564d5109 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:52.634667 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4b9f8a-9120-44af-ba1c-5028564d5109 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:52.646542 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 41/80] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:00:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:00:52.647813 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4b9f8a-9120-44af-ba1c-5028564d5109 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:52.665275 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:52.689156 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 40/81] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:00:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 23:00:52.690191 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae9672-a6eb-45b7-ad3e-f29462ef2a22 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:52.783439 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:52.794214 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4b9f8a-9120-44af-ba1c-5028564d5109 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:52.839231 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae9672-a6eb-45b7-ad3e-f29462ef2a22 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:52.849669 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a8aaa-01d2-49b5-8af6-70b68b256b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:52.850699 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a8aaa-01d2-49b5-8af6-70b68b256b1e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:52.851149 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a8aaa-01d2-49b5-8af6-70b68b256b1e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:52.851634 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a8aaa-01d2-49b5-8af6-70b68b256b1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:52.891961 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb75262-e3b1-438a-8436-a8e18ef65160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:52.892754 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb75262-e3b1-438a-8436-a8e18ef65160 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:52.893095 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb75262-e3b1-438a-8436-a8e18ef65160 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:52.893561 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb75262-e3b1-438a-8436-a8e18ef65160 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:52.895069 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb75262-e3b1-438a-8436-a8e18ef65160 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:52.896989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb75262-e3b1-438a-8436-a8e18ef65160 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:52.899229 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 41/82] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:00:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:00:52.900179 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb75262-e3b1-438a-8436-a8e18ef65160 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:52.952162 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0a8aaa-01d2-49b5-8af6-70b68b256b1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:52.953843 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0a8aaa-01d2-49b5-8af6-70b68b256b1e None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:52.977564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:53.038082 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb75262-e3b1-438a-8436-a8e18ef65160 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:53.082828 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:53.086282 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a0248-4807-4191-bbcb-564728d13b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:53.087182 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a0248-4807-4191-bbcb-564728d13b95 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:53.087572 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a0248-4807-4191-bbcb-564728d13b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:53.087939 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a0248-4807-4191-bbcb-564728d13b95 None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:53.089441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477a0248-4807-4191-bbcb-564728d13b95 None admin] RBAC: Adding query filter params (name=member) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:00:53.089857 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477a0248-4807-4191-bbcb-564728d13b95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:53.091414 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477a0248-4807-4191-bbcb-564728d13b95 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:53.093968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 42/83] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:00:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 23:00:53.095680 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a8aaa-01d2-49b5-8af6-70b68b256b1e None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:53.108948 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 42/84] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jul 13 23:00:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:00:53.110375 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a0248-4807-4191-bbcb-564728d13b95 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:53.230810 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a8aaa-01d2-49b5-8af6-70b68b256b1e None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:53.246850 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a0248-4807-4191-bbcb-564728d13b95 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:53.268070 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:53.274115 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16e15c-9587-41b9-a390-205e2e28c619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:53.275190 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16e15c-9587-41b9-a390-205e2e28c619 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:53.275629 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16e15c-9587-41b9-a390-205e2e28c619 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:53.276013 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16e15c-9587-41b9-a390-205e2e28c619 None admin] PATH_INFO: `/v3/users/9159a16d47ec45bd861706097b1a5a73` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:53.278613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d16e15c-9587-41b9-a390-205e2e28c619 None admin] RBAC: Authorizing `identity:get_user(user_id=9159a16d47ec45bd861706097b1a5a73)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:53.280204 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d16e15c-9587-41b9-a390-205e2e28c619 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:53.282403 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 43/85] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:00:53 2022] GET /identity/v3/users/9159a16d47ec45bd861706097b1a5a73 => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:00:53.283461 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16e15c-9587-41b9-a390-205e2e28c619 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:53.292686 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf9f18-81c9-428e-a489-134b7b2bc65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:53.293654 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf9f18-81c9-428e-a489-134b7b2bc65e None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:53.294058 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf9f18-81c9-428e-a489-134b7b2bc65e None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:53.294439 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf9f18-81c9-428e-a489-134b7b2bc65e None admin] PATH_INFO: `/v3/projects/bd646bf2ad574160a5fa8742f70da34c` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:53.295883 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bf9f18-81c9-428e-a489-134b7b2bc65e None admin] RBAC: Authorizing `identity:get_project(project_id=bd646bf2ad574160a5fa8742f70da34c)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:53.297509 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bf9f18-81c9-428e-a489-134b7b2bc65e None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:53.299508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 43/86] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:00:53 2022] GET /identity/v3/projects/bd646bf2ad574160a5fa8742f70da34c => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:00:53.305385 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf9f18-81c9-428e-a489-134b7b2bc65e None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:53.374305 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:53.426259 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16e15c-9587-41b9-a390-205e2e28c619 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:53.458973 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf9f18-81c9-428e-a489-134b7b2bc65e None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:53.484769 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1dddf-a0d8-471b-b169-65ee8a09bb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:53.485738 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1dddf-a0d8-471b-b169-65ee8a09bb13 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:53.486211 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1dddf-a0d8-471b-b169-65ee8a09bb13 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:53.486664 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1dddf-a0d8-471b-b169-65ee8a09bb13 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:53.487962 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe1dddf-a0d8-471b-b169-65ee8a09bb13 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:53.490520 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe1dddf-a0d8-471b-b169-65ee8a09bb13 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:53.515899 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4985a470-ea98-4ba0-b7d1-2502accbd934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:53.516861 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4985a470-ea98-4ba0-b7d1-2502accbd934 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:53.517295 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4985a470-ea98-4ba0-b7d1-2502accbd934 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:53.517698 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4985a470-ea98-4ba0-b7d1-2502accbd934 None admin] PATH_INFO: `/v3/projects/bd646bf2ad574160a5fa8742f70da34c` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:53.519257 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4985a470-ea98-4ba0-b7d1-2502accbd934 None admin] RBAC: Authorizing `identity:get_project(project_id=bd646bf2ad574160a5fa8742f70da34c)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:53.521234 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4985a470-ea98-4ba0-b7d1-2502accbd934 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:53.524042 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 44/87] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:00:53 2022] GET /identity/v3/projects/bd646bf2ad574160a5fa8742f70da34c => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:00:53.524880 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4985a470-ea98-4ba0-b7d1-2502accbd934 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:53.533497 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 44/88] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Jul 13 23:00:53 2022] POST /identity/v3/users => generated 289 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:00:53.534591 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1dddf-a0d8-471b-b169-65ee8a09bb13 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:53.638675 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:53.701436 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1dddf-a0d8-471b-b169-65ee8a09bb13 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:53.710816 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4985a470-ea98-4ba0-b7d1-2502accbd934 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:53.732799 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:53.763987 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695ae54-4a08-445d-b439-7db30119cbf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:53.764991 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695ae54-4a08-445d-b439-7db30119cbf6 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:53.765453 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695ae54-4a08-445d-b439-7db30119cbf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:53.765914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695ae54-4a08-445d-b439-7db30119cbf6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:53.777646 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa3e50-f818-4153-8e30-139f04978b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:53.778567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa3e50-f818-4153-8e30-139f04978b45 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:53.778860 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa3e50-f818-4153-8e30-139f04978b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:53.779141 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa3e50-f818-4153-8e30-139f04978b45 None admin] PATH_INFO: `/v3/users/9159a16d47ec45bd861706097b1a5a73` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:53.780726 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa3e50-f818-4153-8e30-139f04978b45 None admin] RBAC: Authorizing `identity:get_user(user_id=9159a16d47ec45bd861706097b1a5a73)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:53.782246 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa3e50-f818-4153-8e30-139f04978b45 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:53.784426 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 45/89] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:00:53 2022] GET /identity/v3/users/9159a16d47ec45bd861706097b1a5a73 => generated 275 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:00:53.785007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa3e50-f818-4153-8e30-139f04978b45 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:53.896868 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e695ae54-4a08-445d-b439-7db30119cbf6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:53.900251 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e695ae54-4a08-445d-b439-7db30119cbf6 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:53.982694 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa3e50-f818-4153-8e30-139f04978b45 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:54.022078 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:54.052608 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19041b36-fb7e-4574-a532-ac4415f0195a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:54.055765 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19041b36-fb7e-4574-a532-ac4415f0195a None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:54.056519 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19041b36-fb7e-4574-a532-ac4415f0195a None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:54.057261 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19041b36-fb7e-4574-a532-ac4415f0195a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:54.059499 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19041b36-fb7e-4574-a532-ac4415f0195a None admin] RBAC: Adding query filter params (role.id=8cfd0b6e71e14e0983c3eb4d7478e027, scope.project.id=bd646bf2ad574160a5fa8742f70da34c, user.id=7f1cf8ea49bb4e7fb83c77a4a463d6fd) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:00:54.060336 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19041b36-fb7e-4574-a532-ac4415f0195a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:54.064287 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19041b36-fb7e-4574-a532-ac4415f0195a None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:54.077257 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 46/90] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:00:53 2022] GET /identity/v3/role_assignments?user.id=7f1cf8ea49bb4e7fb83c77a4a463d6fd&scope.project.id=bd646bf2ad574160a5fa8742f70da34c&role.id=8cfd0b6e71e14e0983c3eb4d7478e027 => generated 259 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:00:54.079272 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19041b36-fb7e-4574-a532-ac4415f0195a None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:54.123681 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 45/91] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:00:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 23:00:54.129733 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695ae54-4a08-445d-b439-7db30119cbf6 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:54.175900 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:54.389656 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19041b36-fb7e-4574-a532-ac4415f0195a None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:54.421213 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695ae54-4a08-445d-b439-7db30119cbf6 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:54.461865 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f60cc-6c61-4976-a1ca-756dd6dc075b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:54.464488 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f60cc-6c61-4976-a1ca-756dd6dc075b None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:54.468077 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f60cc-6c61-4976-a1ca-756dd6dc075b None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:54.468629 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f60cc-6c61-4976-a1ca-756dd6dc075b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:54.469717 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3f60cc-6c61-4976-a1ca-756dd6dc075b None admin] RBAC: Adding query filter params (role.id=f005bdf39a0743238b3476f6b88b3684, scope.project.id=bd646bf2ad574160a5fa8742f70da34c, user.id=9159a16d47ec45bd861706097b1a5a73) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:00:54.470244 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3f60cc-6c61-4976-a1ca-756dd6dc075b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:54.472246 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3f60cc-6c61-4976-a1ca-756dd6dc075b None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:54.485201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 47/92] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:00:54 2022] GET /identity/v3/role_assignments?user.id=9159a16d47ec45bd861706097b1a5a73&scope.project.id=bd646bf2ad574160a5fa8742f70da34c&role.id=f005bdf39a0743238b3476f6b88b3684 => generated 259 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:00:54.487267 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f60cc-6c61-4976-a1ca-756dd6dc075b None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:54.505378 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d55f45-0cfe-4713-86bd-a03b72ae7b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:54.506450 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d55f45-0cfe-4713-86bd-a03b72ae7b17 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:54.506936 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d55f45-0cfe-4713-86bd-a03b72ae7b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:54.507412 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d55f45-0cfe-4713-86bd-a03b72ae7b17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:54.583155 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:54.644943 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f60cc-6c61-4976-a1ca-756dd6dc075b None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:54.687860 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e0951-5168-405a-9fd2-9e9b6400b1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:54.689162 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e0951-5168-405a-9fd2-9e9b6400b1fc None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:54.689665 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e0951-5168-405a-9fd2-9e9b6400b1fc None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:54.690176 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e0951-5168-405a-9fd2-9e9b6400b1fc None admin] PATH_INFO: `/v3/projects/bd646bf2ad574160a5fa8742f70da34c` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:54.691803 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e0951-5168-405a-9fd2-9e9b6400b1fc None admin] RBAC: Authorizing `identity:get_project(project_id=bd646bf2ad574160a5fa8742f70da34c)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:54.693621 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e0951-5168-405a-9fd2-9e9b6400b1fc None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:54.696197 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 48/93] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:00:54 2022] GET /identity/v3/projects/bd646bf2ad574160a5fa8742f70da34c => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:00:54.703038 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e0951-5168-405a-9fd2-9e9b6400b1fc None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:54.704031 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d55f45-0cfe-4713-86bd-a03b72ae7b17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:54.705561 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d55f45-0cfe-4713-86bd-a03b72ae7b17 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:54.740220 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:54.824116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e0951-5168-405a-9fd2-9e9b6400b1fc None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:54.850612 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:54.874569 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6af2ae4-a940-4056-bcd4-05f41456ecbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:54.875707 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af2ae4-a940-4056-bcd4-05f41456ecbb None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:54.876216 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af2ae4-a940-4056-bcd4-05f41456ecbb None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:54.876706 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af2ae4-a940-4056-bcd4-05f41456ecbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:54.877858 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6af2ae4-a940-4056-bcd4-05f41456ecbb None admin] RBAC: Adding query filter params (role.id=5f0ca8371a394d339621748b570e78ee, scope.project.id=bd646bf2ad574160a5fa8742f70da34c, user.id=9159a16d47ec45bd861706097b1a5a73) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:00:54.878445 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6af2ae4-a940-4056-bcd4-05f41456ecbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:54.880261 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6af2ae4-a940-4056-bcd4-05f41456ecbb None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:54.887937 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 49/94] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:00:54 2022] GET /identity/v3/role_assignments?user.id=9159a16d47ec45bd861706097b1a5a73&scope.project.id=bd646bf2ad574160a5fa8742f70da34c&role.id=5f0ca8371a394d339621748b570e78ee => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:00:54.947426 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 46/95] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:00:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 822 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 23:00:54.956667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6af2ae4-a940-4056-bcd4-05f41456ecbb None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:55.111742 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6af2ae4-a940-4056-bcd4-05f41456ecbb None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:55.154214 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf4105e-604c-4193-bdd0-ed8c3532cfb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:55.155451 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4105e-604c-4193-bdd0-ed8c3532cfb0 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:55.156000 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4105e-604c-4193-bdd0-ed8c3532cfb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:55.156483 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4105e-604c-4193-bdd0-ed8c3532cfb0 None admin] PATH_INFO: `/v3/projects/dc55a422c9604f3bbfcbed15a01b95ce` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:55.157966 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf4105e-604c-4193-bdd0-ed8c3532cfb0 None admin] RBAC: Authorizing `identity:get_project(project_id=dc55a422c9604f3bbfcbed15a01b95ce)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:55.159821 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf4105e-604c-4193-bdd0-ed8c3532cfb0 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:55.162100 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 50/96] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:00:54 2022] GET /identity/v3/projects/dc55a422c9604f3bbfcbed15a01b95ce => generated 312 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:00:55.172816 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d55f45-0cfe-4713-86bd-a03b72ae7b17 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:55.413815 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d55f45-0cfe-4713-86bd-a03b72ae7b17 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:55.458934 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:55.495868 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1a32b5-afb3-4f42-b55c-e3fc9bc7218b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:55.496934 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a32b5-afb3-4f42-b55c-e3fc9bc7218b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:55.497452 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a32b5-afb3-4f42-b55c-e3fc9bc7218b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:55.497931 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a32b5-afb3-4f42-b55c-e3fc9bc7218b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:55.499272 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1a32b5-afb3-4f42-b55c-e3fc9bc7218b None admin] RBAC: Adding query filter params (role.id=f005bdf39a0743238b3476f6b88b3684, scope.project.id=dc55a422c9604f3bbfcbed15a01b95ce, user.id=9159a16d47ec45bd861706097b1a5a73) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:00:55.499889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1a32b5-afb3-4f42-b55c-e3fc9bc7218b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:55.501642 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1a32b5-afb3-4f42-b55c-e3fc9bc7218b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:55.514882 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 47/97] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:00:55 2022] GET /identity/v3/role_assignments?user.id=9159a16d47ec45bd861706097b1a5a73&scope.project.id=dc55a422c9604f3bbfcbed15a01b95ce&role.id=f005bdf39a0743238b3476f6b88b3684 => generated 259 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:00:56.855867 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c46ed-6cd2-40ff-aa9d-bb2e7b69b266 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:56.856987 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c46ed-6cd2-40ff-aa9d-bb2e7b69b266 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:56.857480 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c46ed-6cd2-40ff-aa9d-bb2e7b69b266 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:56.859135 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 51/98] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:00:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:00:56.865465 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582ccac7-2ac6-43a8-ae3f-8218932b0d70 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:56.866429 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582ccac7-2ac6-43a8-ae3f-8218932b0d70 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:56.866909 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582ccac7-2ac6-43a8-ae3f-8218932b0d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:56.884854 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-582ccac7-2ac6-43a8-ae3f-8218932b0d70 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:00:56.886594 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-582ccac7-2ac6-43a8-ae3f-8218932b0d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:00:56.967879 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8f241-0b74-4b1a-95fc-fa5a2d42e63a None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:56.968522 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8f241-0b74-4b1a-95fc-fa5a2d42e63a None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:56.969004 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8f241-0b74-4b1a-95fc-fa5a2d42e63a None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:56.970605 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 52/99] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:00:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:00:56.974852 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 48/100] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:00:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:00:56.976678 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e95dfa-99b9-4dd8-a0c8-beabd92f6e90 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:56.977186 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e95dfa-99b9-4dd8-a0c8-beabd92f6e90 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:56.977693 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e95dfa-99b9-4dd8-a0c8-beabd92f6e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:57.004716 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-30e95dfa-99b9-4dd8-a0c8-beabd92f6e90 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:00:57.006292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-30e95dfa-99b9-4dd8-a0c8-beabd92f6e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:00:57.012074 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582ccac7-2ac6-43a8-ae3f-8218932b0d70 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:57.100644 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:57.150118 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 53/101] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:00:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:00:57.190480 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e95dfa-99b9-4dd8-a0c8-beabd92f6e90 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:57.231740 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582ccac7-2ac6-43a8-ae3f-8218932b0d70 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:57.261263 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:57.307655 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f0ab3-4bed-47bd-9128-1393b425c3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:57.308849 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f0ab3-4bed-47bd-9128-1393b425c3e2 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:57.309468 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f0ab3-4bed-47bd-9128-1393b425c3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:57.310565 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f0ab3-4bed-47bd-9128-1393b425c3e2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:57.331631 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448f0ab3-4bed-47bd-9128-1393b425c3e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:00:57.348310 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-448f0ab3-4bed-47bd-9128-1393b425c3e2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 23:00:57.348310 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:00:57.348310 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:00:57.348310 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:00:57.348310 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:00:57.348310 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:00:57.348310 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:00:57.348310 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:00:57.348310 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:00:57.348310 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:00:57.348310 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:00:57.348310 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:00:57.348310 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:00:57.348310 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:00:57.348310 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:00:57.348310 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:00:57.349247 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:00:57.349247 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:00:57.349247 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:00:57.349247 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:00:57.349247 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:00:57.349247 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:00:57.349247 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:00:57.349247 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:00:57.349247 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:00:57.349247 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:00:57.349247 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:00:57.349247 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:00:57.349247 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:00:57.349247 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:00:57.349247 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:00:57.349247 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:00:57.350095 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:00:57.350095 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:00:57.350095 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:00:57.350095 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:00:57.350095 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:00:57.350095 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:00:57.350095 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:00:57.350095 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:00:57.350095 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:00:57.350095 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:00:57.350095 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 23:00:57.350095 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:00:57.352682 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 49/102] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:00:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:00:57.354487 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6fd40-8b3b-437b-af2d-bf97645ac2ad None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:57.354986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6fd40-8b3b-437b-af2d-bf97645ac2ad None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:57.355525 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6fd40-8b3b-437b-af2d-bf97645ac2ad None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:57.357185 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 50/103] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:00:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:00:57.362956 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6b7a7-68b0-480e-99cb-c691e899e0a3 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:57.363582 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6b7a7-68b0-480e-99cb-c691e899e0a3 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:57.364141 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6b7a7-68b0-480e-99cb-c691e899e0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:57.383213 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e95dfa-99b9-4dd8-a0c8-beabd92f6e90 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:57.391697 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-53f6b7a7-68b0-480e-99cb-c691e899e0a3 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:00:57.393401 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-53f6b7a7-68b0-480e-99cb-c691e899e0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:00:57.421389 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b2917-0db4-4fe1-98ce-01b11e658482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:57.422115 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b2917-0db4-4fe1-98ce-01b11e658482 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:57.422401 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b2917-0db4-4fe1-98ce-01b11e658482 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:57.422679 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b2917-0db4-4fe1-98ce-01b11e658482 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:57.424106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4b2917-0db4-4fe1-98ce-01b11e658482 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:57.425623 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4b2917-0db4-4fe1-98ce-01b11e658482 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:57.427741 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 54/104] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:00:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:00:57.428333 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b2917-0db4-4fe1-98ce-01b11e658482 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:57.488771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:57.511213 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 51/105] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:00:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 23:00:57.512666 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6b7a7-68b0-480e-99cb-c691e899e0a3 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:57.549141 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b2917-0db4-4fe1-98ce-01b11e658482 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:57.588098 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb587f86-542b-44b0-baa1-64a052e1a0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:57.588819 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb587f86-542b-44b0-baa1-64a052e1a0c2 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:57.589110 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb587f86-542b-44b0-baa1-64a052e1a0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:57.589390 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb587f86-542b-44b0-baa1-64a052e1a0c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:57.591734 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb587f86-542b-44b0-baa1-64a052e1a0c2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:00:57.591734 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb587f86-542b-44b0-baa1-64a052e1a0c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:57.592186 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb587f86-542b-44b0-baa1-64a052e1a0c2 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:57.601315 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 55/106] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jul 13 23:00:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:00:57.602273 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb587f86-542b-44b0-baa1-64a052e1a0c2 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:57.674719 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6b7a7-68b0-480e-99cb-c691e899e0a3 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:57.676332 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:57.717637 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb587f86-542b-44b0-baa1-64a052e1a0c2 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:57.739988 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:57.756368 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c705fc-8150-4272-9511-771e1b82f885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:57.757068 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c705fc-8150-4272-9511-771e1b82f885 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:57.757356 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c705fc-8150-4272-9511-771e1b82f885 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:57.757637 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c705fc-8150-4272-9511-771e1b82f885 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:57.758485 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c705fc-8150-4272-9511-771e1b82f885 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:57.759127 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564c848-6a3f-4c9d-97e8-49c12db86d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:57.760238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c705fc-8150-4272-9511-771e1b82f885 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:57.761224 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564c848-6a3f-4c9d-97e8-49c12db86d72 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:57.761497 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564c848-6a3f-4c9d-97e8-49c12db86d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:57.762053 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564c848-6a3f-4c9d-97e8-49c12db86d72 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:57.774752 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6564c848-6a3f-4c9d-97e8-49c12db86d72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:00:57.787784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-6564c848-6a3f-4c9d-97e8-49c12db86d72 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:00:57.787784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:00:57.787784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:00:57.787784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:00:57.787784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:00:57.787784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:00:57.787784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:00:57.787784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:00:57.787784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:00:57.787784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:00:57.787784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:00:57.787784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:00:57.787784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:00:57.787784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:00:57.787784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:00:57.787784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:00:57.788702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:00:57.788702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:00:57.788702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:00:57.788702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:00:57.788702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:00:57.788702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:00:57.788702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:00:57.788702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:00:57.788702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:00:57.788702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:00:57.788702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:00:57.788702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:00:57.788702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:00:57.788702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:00:57.788702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:00:57.788702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:00:57.789960 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:00:57.789960 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:00:57.789960 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:00:57.789960 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:00:57.789960 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:00:57.789960 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:00:57.789960 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:00:57.789960 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:00:57.789960 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:00:57.789960 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:00:57.789960 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:00:57.789960 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:00:57.789960 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 56/107] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jul 13 23:00:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:00:57.789960 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564c848-6a3f-4c9d-97e8-49c12db86d72 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:57.826326 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 52/108] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Jul 13 23:00:57 2022] POST /identity/v3/projects => generated 302 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:00:57.827991 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c705fc-8150-4272-9511-771e1b82f885 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:57.981926 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564c848-6a3f-4c9d-97e8-49c12db86d72 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:58.034427 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421cb4e2-37f9-4df6-a336-7a7b5e863938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:58.035777 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421cb4e2-37f9-4df6-a336-7a7b5e863938 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:58.036259 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421cb4e2-37f9-4df6-a336-7a7b5e863938 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:58.037197 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421cb4e2-37f9-4df6-a336-7a7b5e863938 None admin] PATH_INFO: `/v3/users/455903e8e07a4b2bbcf0facbf6f18b14` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:58.060186 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421cb4e2-37f9-4df6-a336-7a7b5e863938 None admin] RBAC: Authorizing `identity:get_user(user_id=455903e8e07a4b2bbcf0facbf6f18b14)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:58.062185 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421cb4e2-37f9-4df6-a336-7a7b5e863938 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:58.064934 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 57/109] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:00:57 2022] GET /identity/v3/users/455903e8e07a4b2bbcf0facbf6f18b14 => generated 289 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:00:58.071059 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421cb4e2-37f9-4df6-a336-7a7b5e863938 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:58.071970 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c705fc-8150-4272-9511-771e1b82f885 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:58.107813 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:58.161938 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c7cea-a7b8-4312-bcc6-6a3b3909a5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:58.162917 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c7cea-a7b8-4312-bcc6-6a3b3909a5ab None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:58.163205 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c7cea-a7b8-4312-bcc6-6a3b3909a5ab None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:58.163483 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c7cea-a7b8-4312-bcc6-6a3b3909a5ab None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:58.164300 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3c7cea-a7b8-4312-bcc6-6a3b3909a5ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:00:58.164615 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3c7cea-a7b8-4312-bcc6-6a3b3909a5ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:58.166064 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3c7cea-a7b8-4312-bcc6-6a3b3909a5ab None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:58.187731 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 53/110] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:00:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:00:58.193581 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c7cea-a7b8-4312-bcc6-6a3b3909a5ab None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:58.232941 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:58.278328 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421cb4e2-37f9-4df6-a336-7a7b5e863938 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:58.353395 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ffc525-eab6-4cd8-b895-4b0bec308134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:58.354085 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ffc525-eab6-4cd8-b895-4b0bec308134 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:58.359421 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ffc525-eab6-4cd8-b895-4b0bec308134 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:58.359421 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ffc525-eab6-4cd8-b895-4b0bec308134 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:58.389686 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c7cea-a7b8-4312-bcc6-6a3b3909a5ab None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:58.429999 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd3c20-1ff7-4a8b-b702-2f41e30ce8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:58.431358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd3c20-1ff7-4a8b-b702-2f41e30ce8ae None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:58.431896 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd3c20-1ff7-4a8b-b702-2f41e30ce8ae None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:58.432935 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd3c20-1ff7-4a8b-b702-2f41e30ce8ae None admin] PATH_INFO: `/v3/users/7f1cf8ea49bb4e7fb83c77a4a463d6fd` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:58.434476 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbd3c20-1ff7-4a8b-b702-2f41e30ce8ae None admin] RBAC: Authorizing `identity:get_user(user_id=7f1cf8ea49bb4e7fb83c77a4a463d6fd)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:58.436414 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbd3c20-1ff7-4a8b-b702-2f41e30ce8ae None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:58.438489 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 54/111] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:00:58 2022] GET /identity/v3/users/7f1cf8ea49bb4e7fb83c77a4a463d6fd => generated 247 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:00:58.440201 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1186699-cc80-4add-87b2-4f2564bb1fe7 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:58.440736 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1186699-cc80-4add-87b2-4f2564bb1fe7 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:58.441231 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1186699-cc80-4add-87b2-4f2564bb1fe7 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:58.442873 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 55/112] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:00:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:00:58.444081 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1186699-cc80-4add-87b2-4f2564bb1fe7 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:58.494185 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ffc525-eab6-4cd8-b895-4b0bec308134 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:58.495928 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ffc525-eab6-4cd8-b895-4b0bec308134 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:58.543914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:58.564168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1186699-cc80-4add-87b2-4f2564bb1fe7 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:58.572357 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:58.650343 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d6843-c8bf-4943-9c50-76869e75f043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:58.652438 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d6843-c8bf-4943-9c50-76869e75f043 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:58.653221 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d6843-c8bf-4943-9c50-76869e75f043 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:58.653972 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d6843-c8bf-4943-9c50-76869e75f043 None admin] PATH_INFO: `/v3/projects/bd646bf2ad574160a5fa8742f70da34c` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:58.656428 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d6843-c8bf-4943-9c50-76869e75f043 None admin] RBAC: Authorizing `identity:get_project(project_id=bd646bf2ad574160a5fa8742f70da34c)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:58.659489 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d6843-c8bf-4943-9c50-76869e75f043 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:58.663321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 56/113] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:00:58 2022] GET /identity/v3/projects/bd646bf2ad574160a5fa8742f70da34c => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:00:58.665816 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d599642-4328-40ac-82a8-f7ef004101e2 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:58.666335 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d599642-4328-40ac-82a8-f7ef004101e2 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:58.667377 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d599642-4328-40ac-82a8-f7ef004101e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:58.689584 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-3d599642-4328-40ac-82a8-f7ef004101e2 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:00:58.690815 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3d599642-4328-40ac-82a8-f7ef004101e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:00:58.697343 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 58/114] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:00:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 622 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 23:00:58.697343 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7eace-343a-4d0b-a8cf-b659ef68d1a2 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:58.697343 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7eace-343a-4d0b-a8cf-b659ef68d1a2 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:58.697343 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7eace-343a-4d0b-a8cf-b659ef68d1a2 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:58.703215 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 59/115] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:00:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:00:58.703215 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a62edd-16f5-4303-a031-0e50ef7ac970 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:58.703215 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a62edd-16f5-4303-a031-0e50ef7ac970 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:58.703215 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a62edd-16f5-4303-a031-0e50ef7ac970 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:58.704936 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 60/116] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:00:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:00:58.706008 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a62edd-16f5-4303-a031-0e50ef7ac970 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:58.802349 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 57/117] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:00:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:00:58.804175 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d599642-4328-40ac-82a8-f7ef004101e2 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:58.835283 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a62edd-16f5-4303-a031-0e50ef7ac970 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:58.865762 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:58.878273 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89278871-20c7-4395-89db-b4e9e98e5e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:58.879175 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89278871-20c7-4395-89db-b4e9e98e5e98 None admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:58.879459 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89278871-20c7-4395-89db-b4e9e98e5e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:58.879895 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89278871-20c7-4395-89db-b4e9e98e5e98 None admin] PATH_INFO: `/v3/projects/bd646bf2ad574160a5fa8742f70da34c/users/7f1cf8ea49bb4e7fb83c77a4a463d6fd/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:58.882073 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89278871-20c7-4395-89db-b4e9e98e5e98 None admin] RBAC: Authorizing `identity:create_grant(project_id=bd646bf2ad574160a5fa8742f70da34c, user_id=7f1cf8ea49bb4e7fb83c77a4a463d6fd, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:58.883578 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89278871-20c7-4395-89db-b4e9e98e5e98 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:58.900449 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 61/118] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jul 13 23:00:58 2022] PUT /identity/v3/projects/bd646bf2ad574160a5fa8742f70da34c/users/7f1cf8ea49bb4e7fb83c77a4a463d6fd/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:00:58.902843 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fcef4d-a3af-4f98-a82f-3cd91de70c24 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:58.903152 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fcef4d-a3af-4f98-a82f-3cd91de70c24 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:58.903438 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fcef4d-a3af-4f98-a82f-3cd91de70c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:58.913135 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:58.931401 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d599642-4328-40ac-82a8-f7ef004101e2 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:58.933907 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-38fcef4d-a3af-4f98-a82f-3cd91de70c24 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:00:58.936199 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-38fcef4d-a3af-4f98-a82f-3cd91de70c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:00:58.971172 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a062004-0d95-49f6-acec-d9f0f09405ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:58.972288 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a062004-0d95-49f6-acec-d9f0f09405ff None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:58.972673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a062004-0d95-49f6-acec-d9f0f09405ff None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:58.972971 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a062004-0d95-49f6-acec-d9f0f09405ff None admin] PATH_INFO: `/v3/projects/bd646bf2ad574160a5fa8742f70da34c` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:58.974329 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a062004-0d95-49f6-acec-d9f0f09405ff None admin] RBAC: Authorizing `identity:get_project(project_id=bd646bf2ad574160a5fa8742f70da34c)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:58.976164 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a062004-0d95-49f6-acec-d9f0f09405ff None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:58.978180 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 58/119] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:00:58 2022] GET /identity/v3/projects/bd646bf2ad574160a5fa8742f70da34c => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:00:58.979997 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5f22d5-0937-4368-9019-b06832fec918 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:58.980286 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5f22d5-0937-4368-9019-b06832fec918 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:58.980570 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5f22d5-0937-4368-9019-b06832fec918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:59.001790 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-bb5f22d5-0937-4368-9019-b06832fec918 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:00:59.003626 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-bb5f22d5-0937-4368-9019-b06832fec918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:00:59.038900 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 62/120] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:00:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:00:59.039800 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fcef4d-a3af-4f98-a82f-3cd91de70c24 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:59.102674 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 59/121] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:00:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:00:59.103488 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5f22d5-0937-4368-9019-b06832fec918 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:59.229842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fcef4d-a3af-4f98-a82f-3cd91de70c24 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:59.259730 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:59.260110 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5f22d5-0937-4368-9019-b06832fec918 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:59.280119 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:59.292691 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be10ac6-d54c-437a-af5e-6f3bf3f57691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:59.294456 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be10ac6-d54c-437a-af5e-6f3bf3f57691 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:59.294956 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be10ac6-d54c-437a-af5e-6f3bf3f57691 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:59.295504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be10ac6-d54c-437a-af5e-6f3bf3f57691 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:59.303553 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ead47-ca53-483e-bc7f-b4ec64df41c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:59.304264 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ead47-ca53-483e-bc7f-b4ec64df41c1 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:59.304557 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ead47-ca53-483e-bc7f-b4ec64df41c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:59.304847 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ead47-ca53-483e-bc7f-b4ec64df41c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:59.305705 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ead47-ca53-483e-bc7f-b4ec64df41c1 None admin] RBAC: Adding query filter params (role.id=2562dfc9fd0845678cb5beff6a715255, scope.project.id=bd646bf2ad574160a5fa8742f70da34c, user.id=455903e8e07a4b2bbcf0facbf6f18b14) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:00:59.306027 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ead47-ca53-483e-bc7f-b4ec64df41c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:59.307606 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ead47-ca53-483e-bc7f-b4ec64df41c1 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:59.313850 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 60/122] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:00:59 2022] GET /identity/v3/role_assignments?user.id=455903e8e07a4b2bbcf0facbf6f18b14&scope.project.id=bd646bf2ad574160a5fa8742f70da34c&role.id=2562dfc9fd0845678cb5beff6a715255 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:00:59.314684 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ead47-ca53-483e-bc7f-b4ec64df41c1 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:59.314981 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be10ac6-d54c-437a-af5e-6f3bf3f57691 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:00:59.337764 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-5be10ac6-d54c-437a-af5e-6f3bf3f57691 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 23:00:59.337764 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:00:59.337764 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:00:59.337764 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:00:59.337764 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:00:59.337764 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:00:59.337764 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:00:59.337764 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:00:59.337764 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:00:59.337764 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:00:59.337764 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:00:59.337764 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:00:59.337764 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:00:59.337764 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:00:59.337764 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:00:59.337764 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:00:59.339821 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:00:59.339821 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:00:59.339821 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:00:59.339821 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:00:59.339821 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:00:59.339821 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:00:59.339821 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:00:59.339821 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:00:59.339821 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:00:59.339821 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:00:59.339821 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:00:59.339821 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:00:59.339821 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:00:59.339821 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:00:59.339821 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:00:59.339821 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:00:59.341145 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:00:59.341145 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:00:59.341145 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:00:59.341145 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:00:59.341145 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:00:59.341145 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:00:59.341145 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:00:59.341145 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:00:59.341145 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:00:59.341145 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:00:59.341145 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 23:00:59.341145 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:00:59.346177 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 63/123] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:00:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:00:59.347349 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be10ac6-d54c-437a-af5e-6f3bf3f57691 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:59.457564 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ead47-ca53-483e-bc7f-b4ec64df41c1 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:59.501767 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa8db7-c64e-428c-a0af-b47f11563a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:59.502593 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa8db7-c64e-428c-a0af-b47f11563a14 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:59.502882 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa8db7-c64e-428c-a0af-b47f11563a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:59.503161 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa8db7-c64e-428c-a0af-b47f11563a14 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:59.516450 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa8db7-c64e-428c-a0af-b47f11563a14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:00:59.524674 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-49aa8db7-c64e-428c-a0af-b47f11563a14 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 23:00:59.524674 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:00:59.524674 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:00:59.524674 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:00:59.524674 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:00:59.524674 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:00:59.524674 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:00:59.524674 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:00:59.524674 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:00:59.524674 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:00:59.524674 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:00:59.524674 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:00:59.524674 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:00:59.524674 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:00:59.524674 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:00:59.524674 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:00:59.527129 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:00:59.527129 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:00:59.527129 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:00:59.527129 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:00:59.527129 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:00:59.527129 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:00:59.527129 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:00:59.527129 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:00:59.527129 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:00:59.527129 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:00:59.527129 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:00:59.527129 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:00:59.527129 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:00:59.527129 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:00:59.527129 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:00:59.527129 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:00:59.528935 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:00:59.528935 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:00:59.528935 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:00:59.528935 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:00:59.528935 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:00:59.528935 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:00:59.528935 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:00:59.528935 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:00:59.528935 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:00:59.528935 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:00:59.528935 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 23:00:59.528935 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:00:59.528935 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 61/124] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:00:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:00:59.528935 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa8db7-c64e-428c-a0af-b47f11563a14 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:59.532019 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:59.648650 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa8db7-c64e-428c-a0af-b47f11563a14 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:59.674572 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be10ac6-d54c-437a-af5e-6f3bf3f57691 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:59.686497 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1255a84-cce1-4e80-8673-85aea33074da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:59.687265 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1255a84-cce1-4e80-8673-85aea33074da None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:59.687634 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1255a84-cce1-4e80-8673-85aea33074da None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:59.689285 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1255a84-cce1-4e80-8673-85aea33074da None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:59.690092 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1255a84-cce1-4e80-8673-85aea33074da None admin] RBAC: Adding query filter params (name=member) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:00:59.690408 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1255a84-cce1-4e80-8673-85aea33074da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:59.691969 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1255a84-cce1-4e80-8673-85aea33074da None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:59.702864 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 62/125] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jul 13 23:00:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:00:59.703423 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1255a84-cce1-4e80-8673-85aea33074da None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:59.735573 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5809c8d6-5768-41bd-bb19-e2119974cdb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:59.736942 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5809c8d6-5768-41bd-bb19-e2119974cdb8 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:59.737435 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5809c8d6-5768-41bd-bb19-e2119974cdb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:59.737927 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5809c8d6-5768-41bd-bb19-e2119974cdb8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:59.757373 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5809c8d6-5768-41bd-bb19-e2119974cdb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:00:59.774410 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-5809c8d6-5768-41bd-bb19-e2119974cdb8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 23:00:59.774410 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:00:59.774410 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:00:59.774410 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:00:59.774410 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:00:59.774410 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:00:59.774410 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:00:59.774410 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:00:59.774410 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:00:59.774410 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:00:59.774410 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:00:59.774410 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:00:59.774410 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:00:59.774410 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:00:59.774410 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:00:59.774410 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:00:59.776143 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:00:59.776143 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:00:59.776143 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:00:59.776143 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:00:59.776143 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:00:59.776143 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:00:59.776143 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:00:59.776143 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:00:59.776143 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:00:59.776143 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:00:59.776143 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:00:59.776143 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:00:59.776143 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:00:59.776143 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:00:59.776143 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:00:59.776143 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:00:59.779046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:00:59.779046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:00:59.779046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:00:59.779046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:00:59.779046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:00:59.779046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:00:59.779046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:00:59.779046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:00:59.779046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:00:59.779046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:00:59.779046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 23:00:59.779046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:00:59.784433 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:59.784650 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 64/126] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jul 13 23:00:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 430 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:00:59.784650 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5809c8d6-5768-41bd-bb19-e2119974cdb8 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:59.816567 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1255a84-cce1-4e80-8673-85aea33074da None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:59.842264 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:00:59.855891 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e27dbb-f8d1-4e3d-b3fb-a1165049e604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:00:59.857360 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e27dbb-f8d1-4e3d-b3fb-a1165049e604 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:00:59.857626 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e27dbb-f8d1-4e3d-b3fb-a1165049e604 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:00:59.857919 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e27dbb-f8d1-4e3d-b3fb-a1165049e604 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:00:59.858697 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e27dbb-f8d1-4e3d-b3fb-a1165049e604 None admin] RBAC: Adding query filter params (name=member) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:00:59.859115 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e27dbb-f8d1-4e3d-b3fb-a1165049e604 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:00:59.860577 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e27dbb-f8d1-4e3d-b3fb-a1165049e604 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:00:59.870745 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 63/127] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jul 13 23:00:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:00:59.871740 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e27dbb-f8d1-4e3d-b3fb-a1165049e604 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:00:59.971865 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5809c8d6-5768-41bd-bb19-e2119974cdb8 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:00:59.989926 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e27dbb-f8d1-4e3d-b3fb-a1165049e604 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:00.033531 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:00.051566 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747aa68c-0e72-47a9-8a6b-37263ae2c435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:00.051566 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747aa68c-0e72-47a9-8a6b-37263ae2c435 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:00.051566 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747aa68c-0e72-47a9-8a6b-37263ae2c435 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:00.051566 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747aa68c-0e72-47a9-8a6b-37263ae2c435 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:00.052240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747aa68c-0e72-47a9-8a6b-37263ae2c435 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:00.052240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747aa68c-0e72-47a9-8a6b-37263ae2c435 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:00.052240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747aa68c-0e72-47a9-8a6b-37263ae2c435 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:00.059504 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 64/128] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:00:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:00.064472 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747aa68c-0e72-47a9-8a6b-37263ae2c435 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:00.064820 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60313606-e058-4122-9590-77a6ba503e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:00.066263 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60313606-e058-4122-9590-77a6ba503e88 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:00.066748 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60313606-e058-4122-9590-77a6ba503e88 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:00.067229 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60313606-e058-4122-9590-77a6ba503e88 None admin] PATH_INFO: `/v3/users/9159a16d47ec45bd861706097b1a5a73` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:00.072422 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60313606-e058-4122-9590-77a6ba503e88 None admin] RBAC: Authorizing `identity:get_user(user_id=9159a16d47ec45bd861706097b1a5a73)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:00.074549 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60313606-e058-4122-9590-77a6ba503e88 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:00.076566 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 65/129] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:00:59 2022] GET /identity/v3/users/9159a16d47ec45bd861706097b1a5a73 => generated 275 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:00.078499 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60313606-e058-4122-9590-77a6ba503e88 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:00.265234 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747aa68c-0e72-47a9-8a6b-37263ae2c435 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:00.327510 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b8c614-1f55-4a91-891d-11776b2e2418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:00.328961 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8c614-1f55-4a91-891d-11776b2e2418 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:00.329432 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8c614-1f55-4a91-891d-11776b2e2418 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:00.329890 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8c614-1f55-4a91-891d-11776b2e2418 None admin] PATH_INFO: `/v3/users/9159a16d47ec45bd861706097b1a5a73` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:00.336974 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b8c614-1f55-4a91-891d-11776b2e2418 None admin] RBAC: Authorizing `identity:get_user(user_id=9159a16d47ec45bd861706097b1a5a73)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:00.338537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b8c614-1f55-4a91-891d-11776b2e2418 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:00.340368 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 65/130] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:01:00 2022] GET /identity/v3/users/9159a16d47ec45bd861706097b1a5a73 => generated 275 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:00.341169 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b8c614-1f55-4a91-891d-11776b2e2418 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:00.367155 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60313606-e058-4122-9590-77a6ba503e88 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:00.374421 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:00.490799 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f9096-8dfb-4cc4-ad07-0dccf112f5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:00.492347 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f9096-8dfb-4cc4-ad07-0dccf112f5b8 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:00.492858 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f9096-8dfb-4cc4-ad07-0dccf112f5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:00.493370 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f9096-8dfb-4cc4-ad07-0dccf112f5b8 None admin] PATH_INFO: `/v3/users/9159a16d47ec45bd861706097b1a5a73` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:00.499824 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0f9096-8dfb-4cc4-ad07-0dccf112f5b8 None admin] RBAC: Authorizing `identity:get_user(user_id=9159a16d47ec45bd861706097b1a5a73)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:00.502529 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0f9096-8dfb-4cc4-ad07-0dccf112f5b8 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:00.513761 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 66/131] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:01:00 2022] GET /identity/v3/users/9159a16d47ec45bd861706097b1a5a73 => generated 275 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:00.517474 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f9096-8dfb-4cc4-ad07-0dccf112f5b8 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:00.536027 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:00.639983 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f9096-8dfb-4cc4-ad07-0dccf112f5b8 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:00.648391 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b8c614-1f55-4a91-891d-11776b2e2418 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:00.683422 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c4e62-0c30-4fec-84d5-c61278632142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:00.684709 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c4e62-0c30-4fec-84d5-c61278632142 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:00.685322 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c4e62-0c30-4fec-84d5-c61278632142 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:00.685806 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c4e62-0c30-4fec-84d5-c61278632142 None admin] PATH_INFO: `/v3/projects/dc55a422c9604f3bbfcbed15a01b95ce` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:00.688610 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645c4e62-0c30-4fec-84d5-c61278632142 None admin] RBAC: Authorizing `identity:get_project(project_id=dc55a422c9604f3bbfcbed15a01b95ce)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:00.690972 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645c4e62-0c30-4fec-84d5-c61278632142 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:00.694702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 67/132] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:01:00 2022] GET /identity/v3/projects/dc55a422c9604f3bbfcbed15a01b95ce => generated 312 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:00.696806 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c4e62-0c30-4fec-84d5-c61278632142 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:00.741361 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e031b8a-5883-4bbe-969f-1d046f374da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:00.742607 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e031b8a-5883-4bbe-969f-1d046f374da7 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:00.743103 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e031b8a-5883-4bbe-969f-1d046f374da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:00.743595 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e031b8a-5883-4bbe-969f-1d046f374da7 None admin] PATH_INFO: `/v3/projects/bd646bf2ad574160a5fa8742f70da34c` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:00.746205 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e031b8a-5883-4bbe-969f-1d046f374da7 None admin] RBAC: Authorizing `identity:get_project(project_id=bd646bf2ad574160a5fa8742f70da34c)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:00.747974 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e031b8a-5883-4bbe-969f-1d046f374da7 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:00.749992 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 66/133] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:01:00 2022] GET /identity/v3/projects/bd646bf2ad574160a5fa8742f70da34c => generated 298 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:00.753002 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e031b8a-5883-4bbe-969f-1d046f374da7 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:00.792883 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:00.860538 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c4e62-0c30-4fec-84d5-c61278632142 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:00.899418 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb7e909-d6db-4a95-a7f6-237817c0633e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:00.900340 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7e909-d6db-4a95-a7f6-237817c0633e None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:00.900622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7e909-d6db-4a95-a7f6-237817c0633e None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:00.901488 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7e909-d6db-4a95-a7f6-237817c0633e None admin] PATH_INFO: `/v3/projects/bd646bf2ad574160a5fa8742f70da34c` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:00.902824 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb7e909-d6db-4a95-a7f6-237817c0633e None admin] RBAC: Authorizing `identity:get_project(project_id=bd646bf2ad574160a5fa8742f70da34c)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:00.904353 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb7e909-d6db-4a95-a7f6-237817c0633e None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:00.906173 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 68/134] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:01:00 2022] GET /identity/v3/projects/bd646bf2ad574160a5fa8742f70da34c => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:00.907039 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb7e909-d6db-4a95-a7f6-237817c0633e None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:01.020650 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb7e909-d6db-4a95-a7f6-237817c0633e None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:01.048181 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:01.062383 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449452e-4534-4e1c-8dad-5343ce6f1d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:01.063270 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449452e-4534-4e1c-8dad-5343ce6f1d1a None admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:01.063594 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449452e-4534-4e1c-8dad-5343ce6f1d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:01.064106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449452e-4534-4e1c-8dad-5343ce6f1d1a None admin] PATH_INFO: `/v3/projects/bd646bf2ad574160a5fa8742f70da34c/users/9159a16d47ec45bd861706097b1a5a73/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:01.066515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f449452e-4534-4e1c-8dad-5343ce6f1d1a None admin] RBAC: Authorizing `identity:create_grant(project_id=bd646bf2ad574160a5fa8742f70da34c, user_id=9159a16d47ec45bd861706097b1a5a73, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:01.072539 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f449452e-4534-4e1c-8dad-5343ce6f1d1a None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:01.091910 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e031b8a-5883-4bbe-969f-1d046f374da7 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:01.100766 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 69/135] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jul 13 23:01:00 2022] PUT /identity/v3/projects/bd646bf2ad574160a5fa8742f70da34c/users/9159a16d47ec45bd861706097b1a5a73/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:01:01.101672 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449452e-4534-4e1c-8dad-5343ce6f1d1a None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:01.150770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:01.167360 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a22c743-275d-497c-8566-d7391b55ce3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:01.170018 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a22c743-275d-497c-8566-d7391b55ce3d None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:01.170299 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a22c743-275d-497c-8566-d7391b55ce3d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:01.170577 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a22c743-275d-497c-8566-d7391b55ce3d None admin] PATH_INFO: `/v3/projects/dc55a422c9604f3bbfcbed15a01b95ce/users/9159a16d47ec45bd861706097b1a5a73/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:01.173590 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a22c743-275d-497c-8566-d7391b55ce3d None admin] RBAC: Authorizing `identity:create_grant(project_id=dc55a422c9604f3bbfcbed15a01b95ce, user_id=9159a16d47ec45bd861706097b1a5a73, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:01.178029 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a22c743-275d-497c-8566-d7391b55ce3d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:01.206644 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 67/136] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jul 13 23:01:00 2022] PUT /identity/v3/projects/dc55a422c9604f3bbfcbed15a01b95ce/users/9159a16d47ec45bd861706097b1a5a73/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:01:01.277495 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449452e-4534-4e1c-8dad-5343ce6f1d1a None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:01.325964 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46848d11-b4b4-41cc-b28b-56221376ccd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:01.326871 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46848d11-b4b4-41cc-b28b-56221376ccd9 None admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:01.327161 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46848d11-b4b4-41cc-b28b-56221376ccd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:01.327438 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46848d11-b4b4-41cc-b28b-56221376ccd9 None admin] PATH_INFO: `/v3/projects/bd646bf2ad574160a5fa8742f70da34c/users/9159a16d47ec45bd861706097b1a5a73/roles/5f0ca8371a394d339621748b570e78ee` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:01.333310 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46848d11-b4b4-41cc-b28b-56221376ccd9 None admin] RBAC: Authorizing `identity:create_grant(project_id=bd646bf2ad574160a5fa8742f70da34c, user_id=9159a16d47ec45bd861706097b1a5a73, role_id=5f0ca8371a394d339621748b570e78ee)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:01.334859 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46848d11-b4b4-41cc-b28b-56221376ccd9 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:01.366285 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 70/137] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jul 13 23:01:01 2022] PUT /identity/v3/projects/bd646bf2ad574160a5fa8742f70da34c/users/9159a16d47ec45bd861706097b1a5a73/roles/5f0ca8371a394d339621748b570e78ee => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:01:01.459441 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d84f07-381e-48b9-9615-f553a3aeeb4e None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:01.460414 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d84f07-381e-48b9-9615-f553a3aeeb4e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:01.460845 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d84f07-381e-48b9-9615-f553a3aeeb4e None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:01.462519 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 68/138] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:01.469920 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacdd2e4-b815-4321-bfa0-e37f1f02bd61 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:01.470568 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacdd2e4-b815-4321-bfa0-e37f1f02bd61 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:01.471048 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacdd2e4-b815-4321-bfa0-e37f1f02bd61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:01.489960 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-aacdd2e4-b815-4321-bfa0-e37f1f02bd61 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:01.491672 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-aacdd2e4-b815-4321-bfa0-e37f1f02bd61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:01.519801 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:01.571271 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 71/139] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:01.597178 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d84f07-381e-48b9-9615-f553a3aeeb4e None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:01.721228 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d84f07-381e-48b9-9615-f553a3aeeb4e None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:01.758938 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe598733-cd71-46e0-aa29-4b140479bb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:01.759826 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe598733-cd71-46e0-aa29-4b140479bb0c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:01.760116 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe598733-cd71-46e0-aa29-4b140479bb0c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:01.760929 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe598733-cd71-46e0-aa29-4b140479bb0c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:01.762215 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe598733-cd71-46e0-aa29-4b140479bb0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:01.763742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe598733-cd71-46e0-aa29-4b140479bb0c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:01.765744 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 69/140] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:01:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:01.771990 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacdd2e4-b815-4321-bfa0-e37f1f02bd61 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:01.887828 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacdd2e4-b815-4321-bfa0-e37f1f02bd61 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:01.926912 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8de5c8a-bf9b-4441-89b4-9c256c6382b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:01.927658 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de5c8a-bf9b-4441-89b4-9c256c6382b5 None admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:01.928355 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de5c8a-bf9b-4441-89b4-9c256c6382b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:01.928643 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de5c8a-bf9b-4441-89b4-9c256c6382b5 None admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:01.929398 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8de5c8a-bf9b-4441-89b4-9c256c6382b5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:01.930901 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8de5c8a-bf9b-4441-89b4-9c256c6382b5 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:01.958335 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 72/141] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Jul 13 23:01:01 2022] POST /identity/v3/users => generated 283 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:01:02.029189 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad9504f-87cc-4fec-9d78-f9846709f7e7 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:02.029537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad9504f-87cc-4fec-9d78-f9846709f7e7 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:02.029768 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad9504f-87cc-4fec-9d78-f9846709f7e7 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:02.031173 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 70/142] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:02.037190 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efedeba7-843d-4d98-b802-5e8c3642beca None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:02.037494 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efedeba7-843d-4d98-b802-5e8c3642beca None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:02.037757 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efedeba7-843d-4d98-b802-5e8c3642beca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:02.071381 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-efedeba7-843d-4d98-b802-5e8c3642beca None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:02.073106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-efedeba7-843d-4d98-b802-5e8c3642beca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:02.087004 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:02.166780 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 73/143] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 23:01:02.203967 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad9504f-87cc-4fec-9d78-f9846709f7e7 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:02.334386 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:02.430808 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad9504f-87cc-4fec-9d78-f9846709f7e7 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:02.476558 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11871b6c-f044-4153-b81c-addaf3024ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:02.477380 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11871b6c-f044-4153-b81c-addaf3024ec9 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:02.477672 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11871b6c-f044-4153-b81c-addaf3024ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:02.477953 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11871b6c-f044-4153-b81c-addaf3024ec9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:02.493259 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11871b6c-f044-4153-b81c-addaf3024ec9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:02.501871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-11871b6c-f044-4153-b81c-addaf3024ec9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:01:02.501871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:02.501871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:02.501871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:02.501871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:02.501871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:02.501871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:02.501871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:02.501871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:02.501871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:02.501871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:02.501871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:02.501871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:02.501871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:02.501871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:02.501871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:02.502742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:02.502742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:02.502742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:02.502742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:02.502742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:02.502742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:02.502742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:02.502742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:02.502742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:02.502742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:02.502742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:02.502742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:02.502742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:02.502742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:02.502742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:02.502742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:02.503508 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:02.503508 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:02.503508 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:02.503508 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:02.503508 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:02.503508 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:02.503508 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:02.503508 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:02.503508 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:02.503508 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:02.503508 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:01:02.503508 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:02.503508 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 71/144] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jul 13 23:01:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:02.510279 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efedeba7-843d-4d98-b802-5e8c3642beca None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:02.658160 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efedeba7-843d-4d98-b802-5e8c3642beca None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:02.707715 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222ebd36-9d1c-408a-a33f-b8b64ae8ee77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:02.708814 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222ebd36-9d1c-408a-a33f-b8b64ae8ee77 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:02.709289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222ebd36-9d1c-408a-a33f-b8b64ae8ee77 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:02.709769 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222ebd36-9d1c-408a-a33f-b8b64ae8ee77 None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:02.710730 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222ebd36-9d1c-408a-a33f-b8b64ae8ee77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:02.711233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222ebd36-9d1c-408a-a33f-b8b64ae8ee77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:02.712873 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222ebd36-9d1c-408a-a33f-b8b64ae8ee77 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:02.724468 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 74/145] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:01:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:02.732735 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11871b6c-f044-4153-b81c-addaf3024ec9 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:02.876988 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11871b6c-f044-4153-b81c-addaf3024ec9 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:02.900338 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:02.914415 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de17024e-b3e7-40dc-93f2-0092f05cc462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:02.916347 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17024e-b3e7-40dc-93f2-0092f05cc462 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:02.916746 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17024e-b3e7-40dc-93f2-0092f05cc462 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:02.917077 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17024e-b3e7-40dc-93f2-0092f05cc462 None admin] PATH_INFO: `/v3/users/7f1cf8ea49bb4e7fb83c77a4a463d6fd` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:02.918420 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de17024e-b3e7-40dc-93f2-0092f05cc462 None admin] RBAC: Authorizing `identity:get_user(user_id=7f1cf8ea49bb4e7fb83c77a4a463d6fd)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:02.919949 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de17024e-b3e7-40dc-93f2-0092f05cc462 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:02.921667 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 72/146] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:01:02 2022] GET /identity/v3/users/7f1cf8ea49bb4e7fb83c77a4a463d6fd => generated 247 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:02.927685 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222ebd36-9d1c-408a-a33f-b8b64ae8ee77 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:02.951873 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:03.153004 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222ebd36-9d1c-408a-a33f-b8b64ae8ee77 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:03.222360 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448fad9-ff83-4ec0-9abe-3850da7c47ff None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:03.222708 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9403c565-8aba-4c6b-90d7-55ec7004fd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:03.222822 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448fad9-ff83-4ec0-9abe-3850da7c47ff None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:03.222975 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448fad9-ff83-4ec0-9abe-3850da7c47ff None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:03.223624 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9403c565-8aba-4c6b-90d7-55ec7004fd9b None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:03.224040 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9403c565-8aba-4c6b-90d7-55ec7004fd9b None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:03.224556 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 73/147] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:01:03.224658 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9403c565-8aba-4c6b-90d7-55ec7004fd9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:03.230315 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59f4d3-1ac7-401a-97ab-21b59aaefc00 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:03.230611 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59f4d3-1ac7-401a-97ab-21b59aaefc00 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:03.230892 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59f4d3-1ac7-401a-97ab-21b59aaefc00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:03.261903 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-dc59f4d3-1ac7-401a-97ab-21b59aaefc00 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:03.263193 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-dc59f4d3-1ac7-401a-97ab-21b59aaefc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:03.344128 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9403c565-8aba-4c6b-90d7-55ec7004fd9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:03.346093 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9403c565-8aba-4c6b-90d7-55ec7004fd9b None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:03.360373 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:03.377504 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 74/148] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 23:01:03.413957 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59f4d3-1ac7-401a-97ab-21b59aaefc00 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:03.477987 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 75/149] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:01:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 23:01:03.485196 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9403c565-8aba-4c6b-90d7-55ec7004fd9b None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:03.561895 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc59f4d3-1ac7-401a-97ab-21b59aaefc00 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:03.578813 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:03.606921 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9403c565-8aba-4c6b-90d7-55ec7004fd9b None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:03.607106 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5034d5-f61f-4b60-a91d-65a78053e653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:03.607896 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5034d5-f61f-4b60-a91d-65a78053e653 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:03.608181 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5034d5-f61f-4b60-a91d-65a78053e653 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:03.608457 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5034d5-f61f-4b60-a91d-65a78053e653 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:03.616427 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:03.624807 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5034d5-f61f-4b60-a91d-65a78053e653 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:03.632728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-bf5034d5-f61f-4b60-a91d-65a78053e653 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 23:01:03.632728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:03.632728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:03.632728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:03.632728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:03.632728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:03.632728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:03.632728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:03.632728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:03.632728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:03.632728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:03.632728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:03.632728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:03.632728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:03.632728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:03.632728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:03.633627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:03.633627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:03.633627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:03.633627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:03.633627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:03.633627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:03.633627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:03.633627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:03.633627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:03.633627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:03.633627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:03.633627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:03.633627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:03.633627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:03.633627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:03.633627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:03.634383 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:03.634383 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:03.634383 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:03.634383 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:03.634383 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:03.634383 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:03.634383 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:03.634383 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:03.634383 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:03.634383 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:03.634383 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 23:01:03.634383 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:03.634383 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 75/150] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:01:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:01:03.640371 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5034d5-f61f-4b60-a91d-65a78053e653 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:03.653520 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f5a116-3336-4c36-8686-1023bcc38622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:03.654905 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5a116-3336-4c36-8686-1023bcc38622 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:03.655463 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5a116-3336-4c36-8686-1023bcc38622 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:03.655960 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5a116-3336-4c36-8686-1023bcc38622 None admin] PATH_INFO: `/v3/projects/bd646bf2ad574160a5fa8742f70da34c` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:03.657440 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f5a116-3336-4c36-8686-1023bcc38622 None admin] RBAC: Authorizing `identity:get_project(project_id=bd646bf2ad574160a5fa8742f70da34c)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:03.659141 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f5a116-3336-4c36-8686-1023bcc38622 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:03.661262 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 76/151] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:01:03 2022] GET /identity/v3/projects/bd646bf2ad574160a5fa8742f70da34c => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:01:03.668144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f5a116-3336-4c36-8686-1023bcc38622 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:03.772231 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5034d5-f61f-4b60-a91d-65a78053e653 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:03.795212 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f5a116-3336-4c36-8686-1023bcc38622 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:03.808806 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b64260e-f0e8-4596-bbcb-4d5a0e2c68c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:03.809827 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64260e-f0e8-4596-bbcb-4d5a0e2c68c8 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:03.810289 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64260e-f0e8-4596-bbcb-4d5a0e2c68c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:03.810739 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64260e-f0e8-4596-bbcb-4d5a0e2c68c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:03.811704 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b64260e-f0e8-4596-bbcb-4d5a0e2c68c8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:03.812215 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b64260e-f0e8-4596-bbcb-4d5a0e2c68c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:03.813811 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b64260e-f0e8-4596-bbcb-4d5a0e2c68c8 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:03.824257 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 76/152] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jul 13 23:01:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:03.830186 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b64260e-f0e8-4596-bbcb-4d5a0e2c68c8 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:03.842892 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec39f89-1cac-4489-9951-db1fee4cc4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:03.843979 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec39f89-1cac-4489-9951-db1fee4cc4e8 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:03.844457 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec39f89-1cac-4489-9951-db1fee4cc4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:03.845021 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec39f89-1cac-4489-9951-db1fee4cc4e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:03.846115 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec39f89-1cac-4489-9951-db1fee4cc4e8 None admin] RBAC: Adding query filter params (role.id=8cfd0b6e71e14e0983c3eb4d7478e027, scope.project.id=bd646bf2ad574160a5fa8742f70da34c, user.id=7f1cf8ea49bb4e7fb83c77a4a463d6fd) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:03.846709 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec39f89-1cac-4489-9951-db1fee4cc4e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:03.848454 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec39f89-1cac-4489-9951-db1fee4cc4e8 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:03.856221 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 77/153] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:01:03 2022] GET /identity/v3/role_assignments?user.id=7f1cf8ea49bb4e7fb83c77a4a463d6fd&scope.project.id=bd646bf2ad574160a5fa8742f70da34c&role.id=8cfd0b6e71e14e0983c3eb4d7478e027 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:03.876227 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:03.961111 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b64260e-f0e8-4596-bbcb-4d5a0e2c68c8 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:03.997995 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a75fc1-431d-45eb-a5b6-7b17d6703493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:03.999274 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a75fc1-431d-45eb-a5b6-7b17d6703493 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:03.999816 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a75fc1-431d-45eb-a5b6-7b17d6703493 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:04.000303 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a75fc1-431d-45eb-a5b6-7b17d6703493 None admin] PATH_INFO: `/v3/users/455903e8e07a4b2bbcf0facbf6f18b14` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:04.001890 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a75fc1-431d-45eb-a5b6-7b17d6703493 None admin] RBAC: Authorizing `identity:get_user(user_id=455903e8e07a4b2bbcf0facbf6f18b14)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:04.003640 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a75fc1-431d-45eb-a5b6-7b17d6703493 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:04.005737 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 77/154] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:01:03 2022] GET /identity/v3/users/455903e8e07a4b2bbcf0facbf6f18b14 => generated 289 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:04.012111 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec39f89-1cac-4489-9951-db1fee4cc4e8 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:04.064341 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:04.137340 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec39f89-1cac-4489-9951-db1fee4cc4e8 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:04.180028 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df8805-2ec8-46a5-a202-8281f382291b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:04.180988 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df8805-2ec8-46a5-a202-8281f382291b None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:04.181194 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df8805-2ec8-46a5-a202-8281f382291b None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:04.181471 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df8805-2ec8-46a5-a202-8281f382291b None admin] PATH_INFO: `/v3/projects/bd646bf2ad574160a5fa8742f70da34c` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:04.185887 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df8805-2ec8-46a5-a202-8281f382291b None admin] RBAC: Authorizing `identity:get_project(project_id=bd646bf2ad574160a5fa8742f70da34c)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:04.187509 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df8805-2ec8-46a5-a202-8281f382291b None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:04.189440 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 78/155] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:01:04 2022] GET /identity/v3/projects/bd646bf2ad574160a5fa8742f70da34c => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:04.195284 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a75fc1-431d-45eb-a5b6-7b17d6703493 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:04.316210 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a75fc1-431d-45eb-a5b6-7b17d6703493 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:04.317882 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:04.359425 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2af0088-e5a2-4d2e-99d2-ae48925262ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:04.359425 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af0088-e5a2-4d2e-99d2-ae48925262ca None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:04.359425 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af0088-e5a2-4d2e-99d2-ae48925262ca None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:04.359425 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af0088-e5a2-4d2e-99d2-ae48925262ca None admin] PATH_INFO: `/v3/projects/bd646bf2ad574160a5fa8742f70da34c/users/455903e8e07a4b2bbcf0facbf6f18b14/roles/2562dfc9fd0845678cb5beff6a715255` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:04.359841 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2af0088-e5a2-4d2e-99d2-ae48925262ca None admin] RBAC: Authorizing `identity:create_grant(project_id=bd646bf2ad574160a5fa8742f70da34c, user_id=455903e8e07a4b2bbcf0facbf6f18b14, role_id=2562dfc9fd0845678cb5beff6a715255)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:04.360231 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2af0088-e5a2-4d2e-99d2-ae48925262ca None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:04.389127 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 78/156] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jul 13 23:01:04 2022] PUT /identity/v3/projects/bd646bf2ad574160a5fa8742f70da34c/users/455903e8e07a4b2bbcf0facbf6f18b14/roles/2562dfc9fd0845678cb5beff6a715255 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:01:04.734627 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519181c-7616-4b92-81d7-f0725807873f None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:04.735012 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519181c-7616-4b92-81d7-f0725807873f None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:04.735285 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8519181c-7616-4b92-81d7-f0725807873f None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:04.736900 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 79/157] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:04.743390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fbf237-5ca7-4482-95e9-dd89be2a41f0 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:04.743903 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fbf237-5ca7-4482-95e9-dd89be2a41f0 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:04.744086 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fbf237-5ca7-4482-95e9-dd89be2a41f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:04.763421 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-55fbf237-5ca7-4482-95e9-dd89be2a41f0 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:04.764684 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-55fbf237-5ca7-4482-95e9-dd89be2a41f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:04.850926 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 79/158] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 23:01:04.886094 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519181c-7616-4b92-81d7-f0725807873f None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:05.019495 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8519181c-7616-4b92-81d7-f0725807873f None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:05.028146 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:05.057636 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a8899-acd8-49c7-936a-9b2af6def1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:05.058552 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a8899-acd8-49c7-936a-9b2af6def1be None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:05.058912 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a8899-acd8-49c7-936a-9b2af6def1be None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:05.059191 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a8899-acd8-49c7-936a-9b2af6def1be None admin] PATH_INFO: `/v3/roles/member` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:05.074411 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2a8899-acd8-49c7-936a-9b2af6def1be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:05.082812 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-9e2a8899-acd8-49c7-936a-9b2af6def1be None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 23:01:05.082812 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:05.082812 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:05.082812 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:05.082812 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:05.082812 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:05.082812 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:05.082812 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:05.082812 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:05.082812 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:05.082812 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:05.082812 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:05.082812 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:05.082812 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:05.082812 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:05.082812 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:05.083760 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:05.083760 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:05.083760 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:05.083760 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:05.083760 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:05.083760 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:05.083760 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:05.083760 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:05.083760 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:05.083760 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:05.083760 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:05.083760 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:05.083760 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:05.083760 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:05.083760 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:05.083760 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:05.084539 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:05.084539 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:05.084539 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:05.084539 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:05.084539 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:05.084539 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:05.084539 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:05.084539 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:05.084539 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:05.084539 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:05.084539 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 23:01:05.084539 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:05.084539 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 80/159] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:01:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:05.091464 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fbf237-5ca7-4482-95e9-dd89be2a41f0 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:05.238120 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:05.247860 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fbf237-5ca7-4482-95e9-dd89be2a41f0 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:05.285883 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c6fca-f43b-4cec-89de-cb8cd4d4188f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:05.286476 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c6fca-f43b-4cec-89de-cb8cd4d4188f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:05.286753 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c6fca-f43b-4cec-89de-cb8cd4d4188f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:05.287025 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c6fca-f43b-4cec-89de-cb8cd4d4188f None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:05.287809 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42c6fca-f43b-4cec-89de-cb8cd4d4188f None admin] RBAC: Adding query filter params (name=member) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:05.288111 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42c6fca-f43b-4cec-89de-cb8cd4d4188f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:05.289508 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42c6fca-f43b-4cec-89de-cb8cd4d4188f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:05.298531 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 80/160] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jul 13 23:01:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:05.304196 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a8899-acd8-49c7-936a-9b2af6def1be None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:05.412062 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a8899-acd8-49c7-936a-9b2af6def1be None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:05.447172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d5e6d-5229-4cdb-865d-7bfa01438e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:05.448879 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d5e6d-5229-4cdb-865d-7bfa01438e05 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:05.449156 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d5e6d-5229-4cdb-865d-7bfa01438e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:05.449423 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d5e6d-5229-4cdb-865d-7bfa01438e05 None admin] PATH_INFO: `/v3/users/9159a16d47ec45bd861706097b1a5a73` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:05.450699 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1d5e6d-5229-4cdb-865d-7bfa01438e05 None admin] RBAC: Authorizing `identity:get_user(user_id=9159a16d47ec45bd861706097b1a5a73)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:05.452275 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1d5e6d-5229-4cdb-865d-7bfa01438e05 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:05.453957 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 81/161] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:01:05 2022] GET /identity/v3/users/9159a16d47ec45bd861706097b1a5a73 => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:01:05.459670 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c6fca-f43b-4cec-89de-cb8cd4d4188f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:05.503847 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c66bc-29bd-431d-b45f-3b011dc3cb73 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:05.504175 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c66bc-29bd-431d-b45f-3b011dc3cb73 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:05.504399 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c66bc-29bd-431d-b45f-3b011dc3cb73 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:05.505853 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 82/162] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:01:05.511633 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eef82f-f814-4310-875f-107cf221bb50 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:05.511917 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eef82f-f814-4310-875f-107cf221bb50 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:05.512190 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eef82f-f814-4310-875f-107cf221bb50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:05.544084 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-51eef82f-f814-4310-875f-107cf221bb50 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:05.545216 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-51eef82f-f814-4310-875f-107cf221bb50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:05.639934 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:05.673662 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c6fca-f43b-4cec-89de-cb8cd4d4188f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:05.770147 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 83/163] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:05.772408 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e4cdb0-ec00-41d5-adf3-4a23e1792689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:05.773109 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e4cdb0-ec00-41d5-adf3-4a23e1792689 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:05.773557 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e4cdb0-ec00-41d5-adf3-4a23e1792689 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:05.773976 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e4cdb0-ec00-41d5-adf3-4a23e1792689 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:05.809124 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eef82f-f814-4310-875f-107cf221bb50 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:05.809505 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:05.860150 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e4cdb0-ec00-41d5-adf3-4a23e1792689 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:05.861959 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e4cdb0-ec00-41d5-adf3-4a23e1792689 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:06.064009 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 81/164] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:01:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 23:01:06.066282 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f09f800-325d-4671-b854-499ed869d22d None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:06.066754 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f09f800-325d-4671-b854-499ed869d22d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:06.067168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f09f800-325d-4671-b854-499ed869d22d None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:06.073561 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 82/165] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:06.075358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f09f800-325d-4671-b854-499ed869d22d None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:06.140005 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:06.157527 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eef82f-f814-4310-875f-107cf221bb50 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:06.190509 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f09f800-325d-4671-b854-499ed869d22d None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:06.227786 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e0b44-b474-4fea-bdcb-fd0bcf05e282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:06.229757 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e0b44-b474-4fea-bdcb-fd0bcf05e282 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:06.230178 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e0b44-b474-4fea-bdcb-fd0bcf05e282 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:06.230586 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e0b44-b474-4fea-bdcb-fd0bcf05e282 None admin] PATH_INFO: `/v3/projects/bd646bf2ad574160a5fa8742f70da34c` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:06.232028 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5e0b44-b474-4fea-bdcb-fd0bcf05e282 None admin] RBAC: Authorizing `identity:get_project(project_id=bd646bf2ad574160a5fa8742f70da34c)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:06.233900 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5e0b44-b474-4fea-bdcb-fd0bcf05e282 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:06.235911 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 83/166] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:01:06 2022] GET /identity/v3/projects/bd646bf2ad574160a5fa8742f70da34c => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:01:06.238141 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555c47c-67c4-4263-913f-bd96cefe8800 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:06.238596 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555c47c-67c4-4263-913f-bd96cefe8800 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:06.239030 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555c47c-67c4-4263-913f-bd96cefe8800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:06.257983 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-6555c47c-67c4-4263-913f-bd96cefe8800 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:06.259418 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6555c47c-67c4-4263-913f-bd96cefe8800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:06.284273 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e934ef-d1d5-4bab-9ed0-72cce161bcba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:06.285131 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e934ef-d1d5-4bab-9ed0-72cce161bcba None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:06.285386 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e934ef-d1d5-4bab-9ed0-72cce161bcba None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:06.285666 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e934ef-d1d5-4bab-9ed0-72cce161bcba None admin] PATH_INFO: `/v3/roles/member` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:06.333253 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 84/167] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:06.334470 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555c47c-67c4-4263-913f-bd96cefe8800 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:06.340164 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e934ef-d1d5-4bab-9ed0-72cce161bcba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:06.348195 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-b8e934ef-d1d5-4bab-9ed0-72cce161bcba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 23:01:06.348195 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:06.348195 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:06.348195 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:06.348195 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:06.348195 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:06.348195 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:06.348195 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:06.348195 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:06.348195 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:06.348195 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:06.348195 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:06.348195 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:06.348195 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:06.348195 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:06.348195 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:06.349227 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:06.349227 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:06.349227 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:06.349227 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:06.349227 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:06.349227 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:06.349227 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:06.349227 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:06.349227 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:06.349227 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:06.349227 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:06.349227 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:06.349227 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:06.349227 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:06.349227 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:06.349227 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:06.350346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:06.350346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:06.350346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:06.350346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:06.350346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:06.350346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:06.350346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:06.350346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:06.350346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:06.350346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:06.350346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 23:01:06.350346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:06.350346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 84/168] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:01:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 541 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:01:06.365812 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e934ef-d1d5-4bab-9ed0-72cce161bcba None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:06.417218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:06.439571 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:06.453041 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555c47c-67c4-4263-913f-bd96cefe8800 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:06.491290 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59ebbe-9b2f-422e-a97d-c40644588059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:06.492387 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59ebbe-9b2f-422e-a97d-c40644588059 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:06.492826 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59ebbe-9b2f-422e-a97d-c40644588059 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:06.493328 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59ebbe-9b2f-422e-a97d-c40644588059 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:06.494307 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59ebbe-9b2f-422e-a97d-c40644588059 None admin] RBAC: Adding query filter params (role.id=f005bdf39a0743238b3476f6b88b3684, scope.project.id=bd646bf2ad574160a5fa8742f70da34c, user.id=9159a16d47ec45bd861706097b1a5a73) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:06.494802 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59ebbe-9b2f-422e-a97d-c40644588059 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:06.496614 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59ebbe-9b2f-422e-a97d-c40644588059 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:06.503433 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 85/169] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:01:06 2022] GET /identity/v3/role_assignments?user.id=9159a16d47ec45bd861706097b1a5a73&scope.project.id=bd646bf2ad574160a5fa8742f70da34c&role.id=f005bdf39a0743238b3476f6b88b3684 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:06.504700 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59ebbe-9b2f-422e-a97d-c40644588059 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:06.592564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e934ef-d1d5-4bab-9ed0-72cce161bcba None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:06.634825 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59ebbe-9b2f-422e-a97d-c40644588059 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:06.661138 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d9145-b990-4ed6-b9c5-430294835ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:06.661846 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d9145-b990-4ed6-b9c5-430294835ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:06.662242 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d9145-b990-4ed6-b9c5-430294835ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:06.662399 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d9145-b990-4ed6-b9c5-430294835ce2 None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:06.663163 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d9145-b990-4ed6-b9c5-430294835ce2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:06.663475 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d9145-b990-4ed6-b9c5-430294835ce2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:06.665089 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d9145-b990-4ed6-b9c5-430294835ce2 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:06.670701 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:06.683645 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 85/170] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jul 13 23:01:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:06.685809 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d9145-b990-4ed6-b9c5-430294835ce2 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:06.698723 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9c17d-5085-45e5-938f-3a2c349a7860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:06.699914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9c17d-5085-45e5-938f-3a2c349a7860 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:06.700354 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9c17d-5085-45e5-938f-3a2c349a7860 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:06.700765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9c17d-5085-45e5-938f-3a2c349a7860 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:06.713767 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b9c17d-5085-45e5-938f-3a2c349a7860 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:06.722472 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-96b9c17d-5085-45e5-938f-3a2c349a7860 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 23:01:06.722472 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:06.722472 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:06.722472 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:06.722472 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:06.722472 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:06.722472 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:06.722472 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:06.722472 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:06.722472 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:06.722472 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:06.722472 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:06.722472 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:06.722472 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:06.722472 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:06.722472 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:06.723618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:06.723618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:06.723618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:06.723618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:06.723618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:06.723618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:06.723618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:06.723618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:06.723618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:06.723618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:06.723618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:06.723618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:06.723618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:06.723618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:06.723618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:06.723618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:06.724489 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:06.724489 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:06.724489 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:06.724489 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:06.724489 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:06.724489 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:06.724489 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:06.724489 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:06.724489 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:06.724489 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:06.724489 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 23:01:06.724489 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:06.726797 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 86/171] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jul 13 23:01:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:06.745966 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9c17d-5085-45e5-938f-3a2c349a7860 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:06.794593 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:06.810991 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d9145-b990-4ed6-b9c5-430294835ce2 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:06.849530 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e6d5dd-fc69-4750-9941-75ebc1cd0f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:06.850334 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6d5dd-fc69-4750-9941-75ebc1cd0f01 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:06.850622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6d5dd-fc69-4750-9941-75ebc1cd0f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:06.850895 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6d5dd-fc69-4750-9941-75ebc1cd0f01 None admin] PATH_INFO: `/v3/users/9159a16d47ec45bd861706097b1a5a73` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:06.852682 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e6d5dd-fc69-4750-9941-75ebc1cd0f01 None admin] RBAC: Authorizing `identity:get_user(user_id=9159a16d47ec45bd861706097b1a5a73)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:06.854175 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e6d5dd-fc69-4750-9941-75ebc1cd0f01 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:06.859556 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 86/172] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:01:06 2022] GET /identity/v3/users/9159a16d47ec45bd861706097b1a5a73 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:01:06.862270 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e6d5dd-fc69-4750-9941-75ebc1cd0f01 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:06.916011 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9c17d-5085-45e5-938f-3a2c349a7860 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:06.988355 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e6d5dd-fc69-4750-9941-75ebc1cd0f01 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:07.032262 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e7597-d49b-4622-aba4-68e0f991a3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:07.032959 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e7597-d49b-4622-aba4-68e0f991a3e7 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:07.033242 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e7597-d49b-4622-aba4-68e0f991a3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:07.033520 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e7597-d49b-4622-aba4-68e0f991a3e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:07.034323 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e7597-d49b-4622-aba4-68e0f991a3e7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:07.034597 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e7597-d49b-4622-aba4-68e0f991a3e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:07.036022 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e7597-d49b-4622-aba4-68e0f991a3e7 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:07.045910 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40b3112-4551-48e4-974c-b8d26fcf22ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:07.045910 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b3112-4551-48e4-974c-b8d26fcf22ac None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:07.045910 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b3112-4551-48e4-974c-b8d26fcf22ac None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:07.046221 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b3112-4551-48e4-974c-b8d26fcf22ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:07.058957 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 87/173] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:01:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:07.082244 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e7597-d49b-4622-aba4-68e0f991a3e7 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:07.082956 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:07.148595 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:07.168202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40b3112-4551-48e4-974c-b8d26fcf22ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:07.170507 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40b3112-4551-48e4-974c-b8d26fcf22ac None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:07.267280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e7597-d49b-4622-aba4-68e0f991a3e7 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:07.305874 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 87/174] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:01:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 23:01:07.311446 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a30333a-bf7c-400f-8f88-73b7830bfb5e None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:07.311760 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a30333a-bf7c-400f-8f88-73b7830bfb5e None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:07.312040 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a30333a-bf7c-400f-8f88-73b7830bfb5e None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:07.313400 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 88/175] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:07.315739 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a30333a-bf7c-400f-8f88-73b7830bfb5e None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:07.340901 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3604d53-8280-4aa5-b4ad-abfd567b4ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:07.341795 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3604d53-8280-4aa5-b4ad-abfd567b4ceb None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:07.342081 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3604d53-8280-4aa5-b4ad-abfd567b4ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:07.342353 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3604d53-8280-4aa5-b4ad-abfd567b4ceb None admin] PATH_INFO: `/v3/users/9159a16d47ec45bd861706097b1a5a73` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:07.348322 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3604d53-8280-4aa5-b4ad-abfd567b4ceb None admin] RBAC: Authorizing `identity:get_user(user_id=9159a16d47ec45bd861706097b1a5a73)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:07.349794 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3604d53-8280-4aa5-b4ad-abfd567b4ceb None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:07.351687 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 88/176] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:01:07 2022] GET /identity/v3/users/9159a16d47ec45bd861706097b1a5a73 => generated 275 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:01:07.352723 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b372b3-08f4-4cb1-a38f-1dd0ef43d358 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:07.353007 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b372b3-08f4-4cb1-a38f-1dd0ef43d358 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:07.353282 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b372b3-08f4-4cb1-a38f-1dd0ef43d358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:07.383896 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:07.400768 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-90b372b3-08f4-4cb1-a38f-1dd0ef43d358 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:07.401916 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-90b372b3-08f4-4cb1-a38f-1dd0ef43d358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:07.431565 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a30333a-bf7c-400f-8f88-73b7830bfb5e None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:07.468705 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d46c90-47a7-4ed7-9850-2772b38b3935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:07.469583 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d46c90-47a7-4ed7-9850-2772b38b3935 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:07.469976 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d46c90-47a7-4ed7-9850-2772b38b3935 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:07.470259 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d46c90-47a7-4ed7-9850-2772b38b3935 None admin] PATH_INFO: `/v3/projects/dc55a422c9604f3bbfcbed15a01b95ce` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:07.471691 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d46c90-47a7-4ed7-9850-2772b38b3935 None admin] RBAC: Authorizing `identity:get_project(project_id=dc55a422c9604f3bbfcbed15a01b95ce)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:07.473169 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d46c90-47a7-4ed7-9850-2772b38b3935 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:07.475090 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 89/177] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:01:07 2022] GET /identity/v3/projects/dc55a422c9604f3bbfcbed15a01b95ce => generated 312 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:01:07.480132 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d46c90-47a7-4ed7-9850-2772b38b3935 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:07.513598 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 89/178] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:07.514947 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce44ad6-1f0d-4e40-9a40-a59f71006fee None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:07.515233 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce44ad6-1f0d-4e40-9a40-a59f71006fee None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:07.515505 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce44ad6-1f0d-4e40-9a40-a59f71006fee None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:07.521229 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 90/179] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:07.521871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce44ad6-1f0d-4e40-9a40-a59f71006fee None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:07.579786 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:07.586533 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d46c90-47a7-4ed7-9850-2772b38b3935 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:07.611918 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:07.625532 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80bcac-b0f4-4d92-a249-29ebb8da8442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:07.626753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80bcac-b0f4-4d92-a249-29ebb8da8442 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:07.627040 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80bcac-b0f4-4d92-a249-29ebb8da8442 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:07.627349 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80bcac-b0f4-4d92-a249-29ebb8da8442 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:07.660381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce44ad6-1f0d-4e40-9a40-a59f71006fee None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:07.697988 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020066ef-9804-44fd-ba61-2112ab4bad04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:07.699002 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020066ef-9804-44fd-ba61-2112ab4bad04 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:07.699283 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020066ef-9804-44fd-ba61-2112ab4bad04 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:07.699622 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020066ef-9804-44fd-ba61-2112ab4bad04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:07.701080 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020066ef-9804-44fd-ba61-2112ab4bad04 None admin] RBAC: Adding query filter params (role.id=f005bdf39a0743238b3476f6b88b3684, scope.project.id=dc55a422c9604f3bbfcbed15a01b95ce, user.id=9159a16d47ec45bd861706097b1a5a73) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:07.701392 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020066ef-9804-44fd-ba61-2112ab4bad04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:07.703465 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020066ef-9804-44fd-ba61-2112ab4bad04 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:07.712833 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 91/180] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:01:07 2022] GET /identity/v3/role_assignments?user.id=9159a16d47ec45bd861706097b1a5a73&scope.project.id=dc55a422c9604f3bbfcbed15a01b95ce&role.id=f005bdf39a0743238b3476f6b88b3684 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:07.714170 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00562ba0-a38a-4e16-8f59-3891323b5a74 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:07.714454 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00562ba0-a38a-4e16-8f59-3891323b5a74 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:07.714730 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00562ba0-a38a-4e16-8f59-3891323b5a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:07.736681 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d80bcac-b0f4-4d92-a249-29ebb8da8442 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:07.738519 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d80bcac-b0f4-4d92-a249-29ebb8da8442 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:07.756068 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-00562ba0-a38a-4e16-8f59-3891323b5a74 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:07.757392 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-00562ba0-a38a-4e16-8f59-3891323b5a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:07.898038 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 92/181] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:07.898905 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00562ba0-a38a-4e16-8f59-3891323b5a74 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:07.943681 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:07.953745 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 90/182] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:01:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 23:01:07.955389 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95142db6-a757-4892-9f33-41ede949f3da None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:07.955706 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95142db6-a757-4892-9f33-41ede949f3da None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:07.955985 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95142db6-a757-4892-9f33-41ede949f3da None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:07.957469 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 91/183] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:07.958170 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95142db6-a757-4892-9f33-41ede949f3da None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:07.992488 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:08.035787 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00562ba0-a38a-4e16-8f59-3891323b5a74 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:08.071434 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef4c55-803e-4d80-971b-dbd95af78388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:08.072299 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef4c55-803e-4d80-971b-dbd95af78388 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:08.072614 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef4c55-803e-4d80-971b-dbd95af78388 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:08.072891 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef4c55-803e-4d80-971b-dbd95af78388 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:08.074134 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95142db6-a757-4892-9f33-41ede949f3da None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:08.083230 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ef4c55-803e-4d80-971b-dbd95af78388 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:08.091466 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-a2ef4c55-803e-4d80-971b-dbd95af78388 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:01:08.091466 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:08.091466 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:08.091466 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:08.091466 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:08.091466 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:08.091466 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:08.091466 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:08.091466 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:08.091466 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:08.091466 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:08.091466 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:08.091466 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:08.091466 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:08.091466 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:08.091466 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:08.092484 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:08.092484 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:08.092484 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:08.092484 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:08.092484 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:08.092484 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:08.092484 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:08.092484 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:08.092484 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:08.092484 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:08.092484 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:08.092484 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:08.092484 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:08.092484 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:08.092484 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:08.092484 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:08.093238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:08.093238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:08.093238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:08.093238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:08.093238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:08.093238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:08.093238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:08.093238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:08.093238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:08.093238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:08.093238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:01:08.093238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:08.093238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 93/184] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jul 13 23:01:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:08.093869 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef4c55-803e-4d80-971b-dbd95af78388 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:08.111696 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c526e-7fef-487f-88ac-f265e48cd7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:08.112506 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c526e-7fef-487f-88ac-f265e48cd7f6 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:08.112786 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c526e-7fef-487f-88ac-f265e48cd7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:08.113058 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c526e-7fef-487f-88ac-f265e48cd7f6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:08.122981 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966c526e-7fef-487f-88ac-f265e48cd7f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:08.132272 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-966c526e-7fef-487f-88ac-f265e48cd7f6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 23:01:08.132272 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:08.132272 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:08.132272 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:08.132272 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:08.132272 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:08.132272 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:08.132272 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:08.132272 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:08.132272 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:08.132272 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:08.132272 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:08.132272 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:08.132272 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:08.132272 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:08.132272 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:08.133202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:08.133202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:08.133202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:08.133202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:08.133202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:08.133202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:08.133202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:08.133202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:08.133202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:08.133202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:08.133202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:08.133202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:08.133202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:08.133202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:08.133202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:08.133202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:08.134190 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:08.134190 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:08.134190 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:08.134190 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:08.134190 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:08.134190 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:08.134190 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:08.134190 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:08.134190 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:08.134190 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:08.134190 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 23:01:08.134190 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:08.134190 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 92/185] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jul 13 23:01:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:08.134987 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d782c-cc75-4749-8436-d1da6e677e55 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:08.134987 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d782c-cc75-4749-8436-d1da6e677e55 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:08.135143 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d782c-cc75-4749-8436-d1da6e677e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:08.152312 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-3a4d782c-cc75-4749-8436-d1da6e677e55 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:08.153526 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-3a4d782c-cc75-4749-8436-d1da6e677e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:08.193324 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:08.203037 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef4c55-803e-4d80-971b-dbd95af78388 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:08.220743 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 93/186] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:08.223196 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956eb2d-6790-43f8-bf2c-e8a70260ccd7 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:08.223488 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956eb2d-6790-43f8-bf2c-e8a70260ccd7 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:08.223799 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956eb2d-6790-43f8-bf2c-e8a70260ccd7 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:08.225116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 94/187] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:08.228316 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3956eb2d-6790-43f8-bf2c-e8a70260ccd7 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:08.244015 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:08.244185 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dabb4f-bc31-4a52-a317-67e92c091960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:08.244990 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dabb4f-bc31-4a52-a317-67e92c091960 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:08.245317 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dabb4f-bc31-4a52-a317-67e92c091960 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:08.245819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dabb4f-bc31-4a52-a317-67e92c091960 None admin] PATH_INFO: `/v3/projects/bd646bf2ad574160a5fa8742f70da34c` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:08.246771 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dabb4f-bc31-4a52-a317-67e92c091960 None admin] RBAC: Authorizing `identity:get_project(project_id=bd646bf2ad574160a5fa8742f70da34c)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:08.248380 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dabb4f-bc31-4a52-a317-67e92c091960 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:08.250100 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 94/188] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:01:08 2022] GET /identity/v3/projects/bd646bf2ad574160a5fa8742f70da34c => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:08.250856 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dabb4f-bc31-4a52-a317-67e92c091960 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:08.332111 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3956eb2d-6790-43f8-bf2c-e8a70260ccd7 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:08.351293 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dabb4f-bc31-4a52-a317-67e92c091960 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:08.370814 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b33bf-e5b9-4ef4-8203-735dd0aceefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:08.371481 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b33bf-e5b9-4ef4-8203-735dd0aceefc None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:08.371786 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b33bf-e5b9-4ef4-8203-735dd0aceefc None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:08.372079 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b33bf-e5b9-4ef4-8203-735dd0aceefc None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:08.372834 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0b33bf-e5b9-4ef4-8203-735dd0aceefc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:08.373137 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0b33bf-e5b9-4ef4-8203-735dd0aceefc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:08.374552 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0b33bf-e5b9-4ef4-8203-735dd0aceefc None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:08.387917 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 95/189] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:01:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:08.387917 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf569e4-ff57-4242-8273-e4fa9cb72301 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:08.387917 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf569e4-ff57-4242-8273-e4fa9cb72301 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:08.387917 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf569e4-ff57-4242-8273-e4fa9cb72301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:08.388707 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144dacf3-0b57-4990-a803-6ee2d49d7a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:08.388707 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144dacf3-0b57-4990-a803-6ee2d49d7a9a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:08.388707 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144dacf3-0b57-4990-a803-6ee2d49d7a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:08.388707 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144dacf3-0b57-4990-a803-6ee2d49d7a9a None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:08.389022 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144dacf3-0b57-4990-a803-6ee2d49d7a9a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:08.389319 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144dacf3-0b57-4990-a803-6ee2d49d7a9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:08.390708 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144dacf3-0b57-4990-a803-6ee2d49d7a9a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:08.399157 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 95/190] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:01:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:08.399994 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144dacf3-0b57-4990-a803-6ee2d49d7a9a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:08.406220 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-aaf569e4-ff57-4242-8273-e4fa9cb72301 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:08.407435 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-aaf569e4-ff57-4242-8273-e4fa9cb72301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:08.422724 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:08.491628 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 96/191] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:08.491628 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf569e4-ff57-4242-8273-e4fa9cb72301 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:08.505531 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:08.511032 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144dacf3-0b57-4990-a803-6ee2d49d7a9a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:08.546203 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f234a-27dd-4f64-964e-faf78c5a00da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:08.547001 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f234a-27dd-4f64-964e-faf78c5a00da None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:08.547285 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f234a-27dd-4f64-964e-faf78c5a00da None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:08.547587 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f234a-27dd-4f64-964e-faf78c5a00da None admin] PATH_INFO: `/v3/domains/default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:08.548767 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0f234a-27dd-4f64-964e-faf78c5a00da None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:08.550220 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0f234a-27dd-4f64-964e-faf78c5a00da None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:08.552136 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 96/192] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:01:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:08.552673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f234a-27dd-4f64-964e-faf78c5a00da None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:08.588489 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf569e4-ff57-4242-8273-e4fa9cb72301 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:08.623049 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a120ea-1ef6-40ad-95ca-269bad19cf97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:08.623728 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a120ea-1ef6-40ad-95ca-269bad19cf97 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:08.624006 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a120ea-1ef6-40ad-95ca-269bad19cf97 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:08.624278 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a120ea-1ef6-40ad-95ca-269bad19cf97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:08.625117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a120ea-1ef6-40ad-95ca-269bad19cf97 None admin] RBAC: Adding query filter params (role.id=5f0ca8371a394d339621748b570e78ee, scope.project.id=bd646bf2ad574160a5fa8742f70da34c, user.id=9159a16d47ec45bd861706097b1a5a73) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:08.625425 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a120ea-1ef6-40ad-95ca-269bad19cf97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:08.626883 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a120ea-1ef6-40ad-95ca-269bad19cf97 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:08.633196 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 97/193] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:01:08 2022] GET /identity/v3/role_assignments?user.id=9159a16d47ec45bd861706097b1a5a73&scope.project.id=bd646bf2ad574160a5fa8742f70da34c&role.id=5f0ca8371a394d339621748b570e78ee => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:08.633829 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a120ea-1ef6-40ad-95ca-269bad19cf97 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:08.644727 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:08.659213 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f234a-27dd-4f64-964e-faf78c5a00da None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:08.700898 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74531670-fdfd-4566-80e9-29959908a222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:08.701877 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74531670-fdfd-4566-80e9-29959908a222 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:08.702157 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74531670-fdfd-4566-80e9-29959908a222 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:08.702423 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74531670-fdfd-4566-80e9-29959908a222 None admin] PATH_INFO: `/v3/users/d2fcecc758494f3aa7df97cae3c0b56d` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:08.715424 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:08.722333 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74531670-fdfd-4566-80e9-29959908a222 None admin] RBAC: Authorizing `identity:get_user(user_id=d2fcecc758494f3aa7df97cae3c0b56d)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:08.723896 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74531670-fdfd-4566-80e9-29959908a222 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:08.726007 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 97/194] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:01:08 2022] GET /identity/v3/users/d2fcecc758494f3aa7df97cae3c0b56d => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:08.726616 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74531670-fdfd-4566-80e9-29959908a222 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:08.760006 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a120ea-1ef6-40ad-95ca-269bad19cf97 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:08.801006 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1128202-baa2-4b29-a389-de4e01877b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:08.801906 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1128202-baa2-4b29-a389-de4e01877b3b None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:08.802192 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1128202-baa2-4b29-a389-de4e01877b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:08.802470 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1128202-baa2-4b29-a389-de4e01877b3b None admin] PATH_INFO: `/v3/users/d2fcecc758494f3aa7df97cae3c0b56d` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:08.803841 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1128202-baa2-4b29-a389-de4e01877b3b None admin] RBAC: Authorizing `identity:get_user(user_id=d2fcecc758494f3aa7df97cae3c0b56d)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:08.805316 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1128202-baa2-4b29-a389-de4e01877b3b None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:08.807059 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 98/195] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:01:08 2022] GET /identity/v3/users/d2fcecc758494f3aa7df97cae3c0b56d => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:08.812447 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1128202-baa2-4b29-a389-de4e01877b3b None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:08.842154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74531670-fdfd-4566-80e9-29959908a222 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:08.881853 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f759c-a746-46ce-8f4c-065ec9e5ef9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:08.882976 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f759c-a746-46ce-8f4c-065ec9e5ef9b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:08.883259 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f759c-a746-46ce-8f4c-065ec9e5ef9b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:08.883615 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f759c-a746-46ce-8f4c-065ec9e5ef9b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:08.894039 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277f759c-a746-46ce-8f4c-065ec9e5ef9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:08.902395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-277f759c-a746-46ce-8f4c-065ec9e5ef9b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 23:01:08.902395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:08.902395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:08.902395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:08.902395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:08.902395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:08.902395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:08.902395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:08.902395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:08.902395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:08.902395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:08.902395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:08.902395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:08.902395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:08.902395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:08.902395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:08.903316 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:08.903316 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:08.903316 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:08.903316 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:08.903316 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:08.903316 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:08.903316 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:08.903316 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:08.903316 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:08.903316 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:08.903316 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:08.903316 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:08.903316 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:08.903316 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:08.903316 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:08.903316 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:08.904361 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:08.904361 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:08.904361 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:08.904361 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:08.904361 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:08.904361 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:08.904361 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:08.904361 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:08.904361 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:08.904361 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:08.904361 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 23:01:08.904361 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:08.904361 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 98/196] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:01:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:08.905554 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f759c-a746-46ce-8f4c-065ec9e5ef9b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:08.913855 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:08.929816 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1128202-baa2-4b29-a389-de4e01877b3b None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:08.962570 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee66d57-d2ed-4423-940d-d487f52c41d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:08.963789 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee66d57-d2ed-4423-940d-d487f52c41d1 None admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:08.964067 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee66d57-d2ed-4423-940d-d487f52c41d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:08.964339 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee66d57-d2ed-4423-940d-d487f52c41d1 None admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:08.965049 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee66d57-d2ed-4423-940d-d487f52c41d1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:08.966529 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee66d57-d2ed-4423-940d-d487f52c41d1 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:08.974695 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:08.988087 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 99/197] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Jul 13 23:01:08 2022] POST /identity/v3/users => generated 297 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:01:08.988988 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee66d57-d2ed-4423-940d-d487f52c41d1 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:09.017887 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f759c-a746-46ce-8f4c-065ec9e5ef9b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:09.058819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4127a28-c7d9-4d11-ae99-79299c25f49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:09.059562 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4127a28-c7d9-4d11-ae99-79299c25f49d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:09.059885 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4127a28-c7d9-4d11-ae99-79299c25f49d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:09.060219 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4127a28-c7d9-4d11-ae99-79299c25f49d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:09.127125 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee66d57-d2ed-4423-940d-d487f52c41d1 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:09.148401 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4127a28-c7d9-4d11-ae99-79299c25f49d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:09.150033 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4127a28-c7d9-4d11-ae99-79299c25f49d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:09.160041 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:09.172429 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762fcca0-21b3-4776-b92d-1f50712c6985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:09.172849 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fcca0-21b3-4776-b92d-1f50712c6985 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:09.173138 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fcca0-21b3-4776-b92d-1f50712c6985 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:09.173419 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fcca0-21b3-4776-b92d-1f50712c6985 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:09.243273 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:09.260864 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762fcca0-21b3-4776-b92d-1f50712c6985 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:09.262408 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762fcca0-21b3-4776-b92d-1f50712c6985 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:09.280386 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 99/198] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:01:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 23:01:09.281172 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4127a28-c7d9-4d11-ae99-79299c25f49d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:09.396618 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 100/199] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:01:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 23:01:09.398223 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762fcca0-21b3-4776-b92d-1f50712c6985 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:09.416569 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4127a28-c7d9-4d11-ae99-79299c25f49d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:09.424517 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:09.455735 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc2f22-966f-4108-b0af-d8c6dcb1f3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:09.456693 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc2f22-966f-4108-b0af-d8c6dcb1f3d7 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:09.457091 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc2f22-966f-4108-b0af-d8c6dcb1f3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:09.457538 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc2f22-966f-4108-b0af-d8c6dcb1f3d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:09.458733 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc2f22-966f-4108-b0af-d8c6dcb1f3d7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:09.459154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc2f22-966f-4108-b0af-d8c6dcb1f3d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:09.461391 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc2f22-966f-4108-b0af-d8c6dcb1f3d7 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:09.472710 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 100/200] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jul 13 23:01:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:09.477908 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc2f22-966f-4108-b0af-d8c6dcb1f3d7 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:09.513173 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:09.524155 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762fcca0-21b3-4776-b92d-1f50712c6985 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:09.573842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fee0f2-5e2f-4b6a-85a2-c73c10bf607c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:09.576604 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fee0f2-5e2f-4b6a-85a2-c73c10bf607c None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:09.577294 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fee0f2-5e2f-4b6a-85a2-c73c10bf607c None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:09.578273 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fee0f2-5e2f-4b6a-85a2-c73c10bf607c None admin] PATH_INFO: `/v3/projects/b64d10f5c7424d909a79d57916eaeac5` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:09.581818 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fee0f2-5e2f-4b6a-85a2-c73c10bf607c None admin] RBAC: Authorizing `identity:get_project(project_id=b64d10f5c7424d909a79d57916eaeac5)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:09.585810 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fee0f2-5e2f-4b6a-85a2-c73c10bf607c None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:09.591709 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 101/201] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:01:09 2022] GET /identity/v3/projects/b64d10f5c7424d909a79d57916eaeac5 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:09.594116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fee0f2-5e2f-4b6a-85a2-c73c10bf607c None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:09.601618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc2f22-966f-4108-b0af-d8c6dcb1f3d7 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:09.655968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2359e946-4e6b-4e03-9258-560845214ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:09.657202 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359e946-4e6b-4e03-9258-560845214ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:09.657491 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359e946-4e6b-4e03-9258-560845214ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:09.657770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359e946-4e6b-4e03-9258-560845214ff6 None admin] PATH_INFO: `/v3/projects/b64d10f5c7424d909a79d57916eaeac5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:09.659017 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2359e946-4e6b-4e03-9258-560845214ff6 None admin] RBAC: Authorizing `identity:get_project(project_id=b64d10f5c7424d909a79d57916eaeac5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:09.661523 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2359e946-4e6b-4e03-9258-560845214ff6 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:09.664060 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 101/202] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:01:09 2022] GET /identity/v3/projects/b64d10f5c7424d909a79d57916eaeac5 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:09.664858 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2359e946-4e6b-4e03-9258-560845214ff6 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:09.715938 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:09.727031 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fee0f2-5e2f-4b6a-85a2-c73c10bf607c None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:09.766531 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b813d7-caec-4a1f-9e79-cdcd78caba3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:09.768302 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b813d7-caec-4a1f-9e79-cdcd78caba3c None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:09.768692 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b813d7-caec-4a1f-9e79-cdcd78caba3c None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:09.769069 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b813d7-caec-4a1f-9e79-cdcd78caba3c None admin] PATH_INFO: `/v3/users/455903e8e07a4b2bbcf0facbf6f18b14` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:09.770926 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b813d7-caec-4a1f-9e79-cdcd78caba3c None admin] RBAC: Authorizing `identity:get_user(user_id=455903e8e07a4b2bbcf0facbf6f18b14)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:09.772586 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b813d7-caec-4a1f-9e79-cdcd78caba3c None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:09.775604 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 102/203] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:01:09 2022] GET /identity/v3/users/455903e8e07a4b2bbcf0facbf6f18b14 => generated 289 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:09.776499 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b813d7-caec-4a1f-9e79-cdcd78caba3c None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:09.793939 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2359e946-4e6b-4e03-9258-560845214ff6 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:09.813343 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:09.841959 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df0c65-aee2-4d1f-8bf2-7e0a2b974bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:09.842539 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df0c65-aee2-4d1f-8bf2-7e0a2b974bea None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:09.842821 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df0c65-aee2-4d1f-8bf2-7e0a2b974bea None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:09.843091 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df0c65-aee2-4d1f-8bf2-7e0a2b974bea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:09.844006 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0df0c65-aee2-4d1f-8bf2-7e0a2b974bea None admin] RBAC: Adding query filter params (role.id=8cfd0b6e71e14e0983c3eb4d7478e027, scope.project.id=b64d10f5c7424d909a79d57916eaeac5, user.id=d2fcecc758494f3aa7df97cae3c0b56d) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:09.844391 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0df0c65-aee2-4d1f-8bf2-7e0a2b974bea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:09.845943 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0df0c65-aee2-4d1f-8bf2-7e0a2b974bea None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:09.851944 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 102/204] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:01:09 2022] GET /identity/v3/role_assignments?user.id=d2fcecc758494f3aa7df97cae3c0b56d&scope.project.id=b64d10f5c7424d909a79d57916eaeac5&role.id=8cfd0b6e71e14e0983c3eb4d7478e027 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:09.860629 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df0c65-aee2-4d1f-8bf2-7e0a2b974bea None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:09.904842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b813d7-caec-4a1f-9e79-cdcd78caba3c None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:09.946498 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb8e3a0-eb67-4e14-adfc-9b8954f4dc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:09.947751 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8e3a0-eb67-4e14-adfc-9b8954f4dc19 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:09.948551 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8e3a0-eb67-4e14-adfc-9b8954f4dc19 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:09.949036 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8e3a0-eb67-4e14-adfc-9b8954f4dc19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:09.950140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb8e3a0-eb67-4e14-adfc-9b8954f4dc19 None admin] RBAC: Adding query filter params (role.id=5f0ca8371a394d339621748b570e78ee, scope.project.id=b64d10f5c7424d909a79d57916eaeac5, user.id=d2fcecc758494f3aa7df97cae3c0b56d) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:09.950678 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb8e3a0-eb67-4e14-adfc-9b8954f4dc19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:09.952468 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb8e3a0-eb67-4e14-adfc-9b8954f4dc19 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:09.959498 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 103/205] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:01:09 2022] GET /identity/v3/role_assignments?user.id=d2fcecc758494f3aa7df97cae3c0b56d&scope.project.id=b64d10f5c7424d909a79d57916eaeac5&role.id=5f0ca8371a394d339621748b570e78ee => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:10.017972 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:10.021338 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df0c65-aee2-4d1f-8bf2-7e0a2b974bea None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:10.063775 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c0e674-6749-4013-911c-064c6e98491f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:10.064569 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0e674-6749-4013-911c-064c6e98491f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:10.064852 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0e674-6749-4013-911c-064c6e98491f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:10.065127 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c0e674-6749-4013-911c-064c6e98491f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:10.156747 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c0e674-6749-4013-911c-064c6e98491f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:10.158273 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c0e674-6749-4013-911c-064c6e98491f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:10.267267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:10.270783 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 103/206] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:01:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 23:01:10.277303 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb8e3a0-eb67-4e14-adfc-9b8954f4dc19 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:10.412557 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:10.424155 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb8e3a0-eb67-4e14-adfc-9b8954f4dc19 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:10.468100 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b237b3e1-c8bf-4222-bb43-5e1e339dc228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:10.468999 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237b3e1-c8bf-4222-bb43-5e1e339dc228 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:10.469280 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237b3e1-c8bf-4222-bb43-5e1e339dc228 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:10.469555 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237b3e1-c8bf-4222-bb43-5e1e339dc228 None admin] PATH_INFO: `/v3/projects/bd646bf2ad574160a5fa8742f70da34c` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:10.470855 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b237b3e1-c8bf-4222-bb43-5e1e339dc228 None admin] RBAC: Authorizing `identity:get_project(project_id=bd646bf2ad574160a5fa8742f70da34c)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:10.472384 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b237b3e1-c8bf-4222-bb43-5e1e339dc228 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:10.474149 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 104/207] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:01:10 2022] GET /identity/v3/projects/bd646bf2ad574160a5fa8742f70da34c => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:10.480754 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c0e674-6749-4013-911c-064c6e98491f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:10.599433 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c0e674-6749-4013-911c-064c6e98491f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:10.634163 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c36bbe-e445-48f8-96d8-dc8869c2645b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:10.635454 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c36bbe-e445-48f8-96d8-dc8869c2645b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:10.635766 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c36bbe-e445-48f8-96d8-dc8869c2645b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:10.636037 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c36bbe-e445-48f8-96d8-dc8869c2645b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:10.636945 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c36bbe-e445-48f8-96d8-dc8869c2645b None admin] RBAC: Adding query filter params (role.id=2562dfc9fd0845678cb5beff6a715255, scope.project.id=bd646bf2ad574160a5fa8742f70da34c, user.id=455903e8e07a4b2bbcf0facbf6f18b14) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:10.637252 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c36bbe-e445-48f8-96d8-dc8869c2645b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:10.638710 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c36bbe-e445-48f8-96d8-dc8869c2645b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:10.651709 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 104/208] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:01:10 2022] GET /identity/v3/role_assignments?user.id=455903e8e07a4b2bbcf0facbf6f18b14&scope.project.id=bd646bf2ad574160a5fa8742f70da34c&role.id=2562dfc9fd0845678cb5beff6a715255 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:11.446378 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d780fc-554e-46f7-bbd5-3742ef6d7e1c None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:11.446932 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d780fc-554e-46f7-bbd5-3742ef6d7e1c None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:11.446932 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d780fc-554e-46f7-bbd5-3742ef6d7e1c None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:11.448927 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 105/209] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:11.454984 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7f6ae-0616-476e-9dc9-6db346b93262 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:11.455981 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7f6ae-0616-476e-9dc9-6db346b93262 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:11.456466 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7f6ae-0616-476e-9dc9-6db346b93262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:11.478359 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c6d7f6ae-0616-476e-9dc9-6db346b93262 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:11.480182 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c6d7f6ae-0616-476e-9dc9-6db346b93262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:11.490797 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d843a6-c6ea-4d6c-b3db-bf3df0d81ee2 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:11.491091 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d843a6-c6ea-4d6c-b3db-bf3df0d81ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:11.491362 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d843a6-c6ea-4d6c-b3db-bf3df0d81ee2 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:11.492951 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 106/210] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:01:11.498807 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4751dd-50d9-4d92-b28e-c986b72ad0ee None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:11.499099 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4751dd-50d9-4d92-b28e-c986b72ad0ee None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:11.499387 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4751dd-50d9-4d92-b28e-c986b72ad0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:11.516777 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-7e4751dd-50d9-4d92-b28e-c986b72ad0ee None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:11.517986 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-7e4751dd-50d9-4d92-b28e-c986b72ad0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:11.543793 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:11.552792 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 105/211] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:11.577393 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d7f6ae-0616-476e-9dc9-6db346b93262 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:11.590866 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 107/212] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 23:01:11.615278 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4751dd-50d9-4d92-b28e-c986b72ad0ee None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:11.680205 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d7f6ae-0616-476e-9dc9-6db346b93262 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:11.723049 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcdd9af-7113-4692-b5dc-48a9b96278c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:11.723974 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcdd9af-7113-4692-b5dc-48a9b96278c2 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:11.724325 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcdd9af-7113-4692-b5dc-48a9b96278c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:11.724697 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcdd9af-7113-4692-b5dc-48a9b96278c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:11.726021 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcdd9af-7113-4692-b5dc-48a9b96278c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:11.726624 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:11.729601 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcdd9af-7113-4692-b5dc-48a9b96278c2 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:11.732733 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 106/213] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:01:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:01:11.738019 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcdd9af-7113-4692-b5dc-48a9b96278c2 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:11.744829 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4751dd-50d9-4d92-b28e-c986b72ad0ee None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:11.779265 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96a6745-bf66-46e9-b595-f0c4131e6650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:11.780289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a6745-bf66-46e9-b595-f0c4131e6650 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:11.780563 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a6745-bf66-46e9-b595-f0c4131e6650 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:11.781097 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a6745-bf66-46e9-b595-f0c4131e6650 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:11.792667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96a6745-bf66-46e9-b595-f0c4131e6650 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:11.800515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-d96a6745-bf66-46e9-b595-f0c4131e6650 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 23:01:11.800515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:11.800515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:11.800515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:11.800515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:11.800515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:11.800515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:11.800515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:11.800515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:11.800515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:11.800515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:11.800515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:11.800515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:11.800515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:11.800515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:11.800515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:11.801416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:11.801416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:11.801416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:11.801416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:11.801416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:11.801416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:11.801416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:11.801416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:11.801416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:11.801416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:11.801416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:11.801416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:11.801416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:11.801416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:11.801416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:11.801416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:11.802203 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:11.802203 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:11.802203 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:11.802203 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:11.802203 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:11.802203 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:11.802203 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:11.802203 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:11.802203 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:11.802203 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:11.802203 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 23:01:11.802203 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:11.802203 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 108/214] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:01:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:11.807887 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96a6745-bf66-46e9-b595-f0c4131e6650 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:11.822380 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:11.868818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcdd9af-7113-4692-b5dc-48a9b96278c2 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:11.905185 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b9131b-8dcf-4d65-ac17-fc2f8d4ba084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:11.905858 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9131b-8dcf-4d65-ac17-fc2f8d4ba084 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:11.906134 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9131b-8dcf-4d65-ac17-fc2f8d4ba084 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:11.906403 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9131b-8dcf-4d65-ac17-fc2f8d4ba084 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:11.907336 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b9131b-8dcf-4d65-ac17-fc2f8d4ba084 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:11.908651 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b9131b-8dcf-4d65-ac17-fc2f8d4ba084 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:11.934054 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 107/215] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Jul 13 23:01:11 2022] POST /identity/v3/users => generated 297 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:01:11.939381 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96a6745-bf66-46e9-b595-f0c4131e6650 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:11.981578 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe9b6f4-69bf-46a1-b4b9-16212898be0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:11.982597 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9b6f4-69bf-46a1-b4b9-16212898be0f None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:11.983086 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9b6f4-69bf-46a1-b4b9-16212898be0f None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:11.983564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9b6f4-69bf-46a1-b4b9-16212898be0f None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:11.984566 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe9b6f4-69bf-46a1-b4b9-16212898be0f None admin] RBAC: Adding query filter params (name=member) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:11.985054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe9b6f4-69bf-46a1-b4b9-16212898be0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:11.986654 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe9b6f4-69bf-46a1-b4b9-16212898be0f None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:11.996826 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 109/216] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jul 13 23:01:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:01:12.003597 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b9131b-8dcf-4d65-ac17-fc2f8d4ba084 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:12.129078 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b9131b-8dcf-4d65-ac17-fc2f8d4ba084 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:12.163670 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:12.184578 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d264a5-c9c4-4d29-be85-96ce0266ca74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:12.185896 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d264a5-c9c4-4d29-be85-96ce0266ca74 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:12.186381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d264a5-c9c4-4d29-be85-96ce0266ca74 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:12.186851 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d264a5-c9c4-4d29-be85-96ce0266ca74 None admin] PATH_INFO: `/v3/users/ea11955c44dd41cc998c143ef68f43d2` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:12.202902 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d264a5-c9c4-4d29-be85-96ce0266ca74 None admin] RBAC: Authorizing `identity:get_user(user_id=ea11955c44dd41cc998c143ef68f43d2)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:12.204809 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d264a5-c9c4-4d29-be85-96ce0266ca74 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:12.207113 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 108/217] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:01:12 2022] GET /identity/v3/users/ea11955c44dd41cc998c143ef68f43d2 => generated 297 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:12.217254 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe9b6f4-69bf-46a1-b4b9-16212898be0f None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:12.234543 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:12.346118 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4abfc66-5264-4dcb-a981-7886c3e487e0 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:12.347065 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4abfc66-5264-4dcb-a981-7886c3e487e0 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:12.347843 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4abfc66-5264-4dcb-a981-7886c3e487e0 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:12.356299 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 109/218] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:12 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:12.368402 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe9b6f4-69bf-46a1-b4b9-16212898be0f None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:12.370481 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919203d-935c-47cf-ba12-7382c57e8431 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:12.370766 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919203d-935c-47cf-ba12-7382c57e8431 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:12.371040 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919203d-935c-47cf-ba12-7382c57e8431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:12.390150 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-9919203d-935c-47cf-ba12-7382c57e8431 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:12.391287 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9919203d-935c-47cf-ba12-7382c57e8431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:12.417102 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d4df4-c813-4aa3-a5b7-bd2cfd315d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:12.417905 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d4df4-c813-4aa3-a5b7-bd2cfd315d39 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:12.418193 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d4df4-c813-4aa3-a5b7-bd2cfd315d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:12.418475 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d4df4-c813-4aa3-a5b7-bd2cfd315d39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:12.482836 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 110/219] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:12.499913 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdb2c0-26a7-41d9-b5a5-8cd3d0ebedd0 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:12.500232 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdb2c0-26a7-41d9-b5a5-8cd3d0ebedd0 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:12.500483 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdb2c0-26a7-41d9-b5a5-8cd3d0ebedd0 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:12.502234 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 111/220] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:01:12.505033 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135d4df4-c813-4aa3-a5b7-bd2cfd315d39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:12.506537 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135d4df4-c813-4aa3-a5b7-bd2cfd315d39 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:12.510622 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beadfbb1-dda1-4866-a728-a9d4a900739e None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:12.511270 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beadfbb1-dda1-4866-a728-a9d4a900739e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:12.511270 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beadfbb1-dda1-4866-a728-a9d4a900739e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:12.531332 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-beadfbb1-dda1-4866-a728-a9d4a900739e None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:12.532944 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-beadfbb1-dda1-4866-a728-a9d4a900739e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:12.540041 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:12.624349 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 112/221] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 23:01:12.625018 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beadfbb1-dda1-4866-a728-a9d4a900739e None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:12.641298 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 110/222] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:01:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 23:01:12.647310 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d4df4-c813-4aa3-a5b7-bd2cfd315d39 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:12.670129 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:12.761089 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beadfbb1-dda1-4866-a728-a9d4a900739e None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:12.769702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d4df4-c813-4aa3-a5b7-bd2cfd315d39 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:12.784747 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:12.807571 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35c6d9-91a7-4fdf-8080-3d28ea8dd448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:12.808468 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35c6d9-91a7-4fdf-8080-3d28ea8dd448 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:12.808749 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35c6d9-91a7-4fdf-8080-3d28ea8dd448 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:12.809021 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35c6d9-91a7-4fdf-8080-3d28ea8dd448 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:12.811290 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76369443-7660-45ee-8aa7-a6e8dc635169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:12.812446 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76369443-7660-45ee-8aa7-a6e8dc635169 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:12.812932 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76369443-7660-45ee-8aa7-a6e8dc635169 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:12.813405 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76369443-7660-45ee-8aa7-a6e8dc635169 None admin] PATH_INFO: `/v3/projects/b64d10f5c7424d909a79d57916eaeac5` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:12.814885 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76369443-7660-45ee-8aa7-a6e8dc635169 None admin] RBAC: Authorizing `identity:get_project(project_id=b64d10f5c7424d909a79d57916eaeac5)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:12.816730 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76369443-7660-45ee-8aa7-a6e8dc635169 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:12.823691 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35c6d9-91a7-4fdf-8080-3d28ea8dd448 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:12.823904 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 111/223] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:01:12 2022] GET /identity/v3/projects/b64d10f5c7424d909a79d57916eaeac5 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:12.824841 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76369443-7660-45ee-8aa7-a6e8dc635169 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:12.830238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-eb35c6d9-91a7-4fdf-8080-3d28ea8dd448 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:01:12.830238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:12.830238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:12.830238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:12.830238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:12.830238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:12.830238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:12.830238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:12.830238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:12.830238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:12.830238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:12.830238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:12.830238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:12.830238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:12.830238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:12.830238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:12.831320 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:12.831320 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:12.831320 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:12.831320 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:12.831320 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:12.831320 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:12.831320 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:12.831320 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:12.831320 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:12.831320 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:12.831320 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:12.831320 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:12.831320 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:12.831320 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:12.831320 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:12.831320 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:12.832168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:12.832168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:12.832168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:12.832168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:12.832168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:12.832168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:12.832168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:12.832168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:12.832168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:12.832168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:12.832168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:01:12.832168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:12.832168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 113/224] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jul 13 23:01:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:12.832809 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35c6d9-91a7-4fdf-8080-3d28ea8dd448 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:12.943659 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76369443-7660-45ee-8aa7-a6e8dc635169 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:12.965436 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:12.973573 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35c6d9-91a7-4fdf-8080-3d28ea8dd448 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:12.988384 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35dca30-9a88-439c-9729-e8101c6be29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:12.989348 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35dca30-9a88-439c-9729-e8101c6be29f None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:12.990044 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35dca30-9a88-439c-9729-e8101c6be29f None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:12.990330 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35dca30-9a88-439c-9729-e8101c6be29f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:13.005539 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35dca30-9a88-439c-9729-e8101c6be29f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:13.011917 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e699d8f-68fe-4337-9009-9dc5969fbc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:13.012699 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e699d8f-68fe-4337-9009-9dc5969fbc70 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:13.013086 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e699d8f-68fe-4337-9009-9dc5969fbc70 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:13.013462 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e699d8f-68fe-4337-9009-9dc5969fbc70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:13.014406 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e699d8f-68fe-4337-9009-9dc5969fbc70 None admin] RBAC: Adding query filter params (role.id=f005bdf39a0743238b3476f6b88b3684, scope.project.id=b64d10f5c7424d909a79d57916eaeac5, user.id=ea11955c44dd41cc998c143ef68f43d2) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:13.014822 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e699d8f-68fe-4337-9009-9dc5969fbc70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:13.015329 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-d35dca30-9a88-439c-9729-e8101c6be29f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 23:01:13.015329 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:13.015329 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:13.015329 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:13.015329 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:13.015329 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:13.015329 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:13.015329 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:13.015329 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:13.015329 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:13.015329 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:13.015329 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:13.015329 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:13.015329 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:13.015329 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:13.015329 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:13.016386 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:13.016386 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:13.016386 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:13.016386 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:13.016386 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:13.016386 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:13.016386 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:13.016386 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:13.016386 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:13.016386 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:13.016386 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:13.016386 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:13.016386 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:13.016386 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:13.016386 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:13.016386 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:13.017667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:13.017667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:13.017667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:13.017667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:13.017667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:13.017667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:13.017667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:13.017667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:13.017667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:13.017667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:13.017667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 23:01:13.017667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:13.017667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 112/225] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jul 13 23:01:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:13.017667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35dca30-9a88-439c-9729-e8101c6be29f None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:13.019480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e699d8f-68fe-4337-9009-9dc5969fbc70 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:13.025603 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 114/226] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:01:12 2022] GET /identity/v3/role_assignments?user.id=ea11955c44dd41cc998c143ef68f43d2&scope.project.id=b64d10f5c7424d909a79d57916eaeac5&role.id=f005bdf39a0743238b3476f6b88b3684 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:13.026568 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e699d8f-68fe-4337-9009-9dc5969fbc70 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:13.071390 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:13.149389 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35dca30-9a88-439c-9729-e8101c6be29f None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:13.192502 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795067d-2ee7-4807-b85a-0a3ef0310f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:13.193347 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795067d-2ee7-4807-b85a-0a3ef0310f13 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:13.193633 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795067d-2ee7-4807-b85a-0a3ef0310f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:13.194085 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795067d-2ee7-4807-b85a-0a3ef0310f13 None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:13.194818 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8795067d-2ee7-4807-b85a-0a3ef0310f13 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:13.195124 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8795067d-2ee7-4807-b85a-0a3ef0310f13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:13.196772 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8795067d-2ee7-4807-b85a-0a3ef0310f13 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:13.207582 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 113/227] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:01:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:13.214406 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795067d-2ee7-4807-b85a-0a3ef0310f13 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:13.303844 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e699d8f-68fe-4337-9009-9dc5969fbc70 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:13.342766 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795067d-2ee7-4807-b85a-0a3ef0310f13 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:13.344894 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:13.350961 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420d7ca1-d183-40fd-bb17-47a804c0f86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:13.351710 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d7ca1-d183-40fd-bb17-47a804c0f86c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:13.351997 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d7ca1-d183-40fd-bb17-47a804c0f86c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:13.352316 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d7ca1-d183-40fd-bb17-47a804c0f86c None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:13.368183 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420d7ca1-d183-40fd-bb17-47a804c0f86c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:13.368183 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420d7ca1-d183-40fd-bb17-47a804c0f86c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:13.368183 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420d7ca1-d183-40fd-bb17-47a804c0f86c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:13.368183 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 115/228] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:01:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:13.380663 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420d7ca1-d183-40fd-bb17-47a804c0f86c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:13.386898 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44457f4-550a-4d74-b29b-6917604ee4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:13.387930 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44457f4-550a-4d74-b29b-6917604ee4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:13.388330 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44457f4-550a-4d74-b29b-6917604ee4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:13.388698 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44457f4-550a-4d74-b29b-6917604ee4f2 None admin] PATH_INFO: `/v3/users/d2fcecc758494f3aa7df97cae3c0b56d` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:13.390010 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44457f4-550a-4d74-b29b-6917604ee4f2 None admin] RBAC: Authorizing `identity:get_user(user_id=d2fcecc758494f3aa7df97cae3c0b56d)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:13.391582 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44457f4-550a-4d74-b29b-6917604ee4f2 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:13.394582 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 114/229] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:01:13 2022] GET /identity/v3/users/d2fcecc758494f3aa7df97cae3c0b56d => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:01:13.401198 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44457f4-550a-4d74-b29b-6917604ee4f2 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:13.447133 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:13.534146 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44457f4-550a-4d74-b29b-6917604ee4f2 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:13.599080 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420d7ca1-d183-40fd-bb17-47a804c0f86c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:13.601795 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abbe486-5915-4204-97ef-54c97b0625f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:13.602936 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abbe486-5915-4204-97ef-54c97b0625f6 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:13.604021 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abbe486-5915-4204-97ef-54c97b0625f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:13.604501 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abbe486-5915-4204-97ef-54c97b0625f6 None admin] PATH_INFO: `/v3/projects/b64d10f5c7424d909a79d57916eaeac5` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:13.606052 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abbe486-5915-4204-97ef-54c97b0625f6 None admin] RBAC: Authorizing `identity:get_project(project_id=b64d10f5c7424d909a79d57916eaeac5)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:13.607859 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abbe486-5915-4204-97ef-54c97b0625f6 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:13.610227 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 115/230] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:01:13 2022] GET /identity/v3/projects/b64d10f5c7424d909a79d57916eaeac5 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:01:13.620607 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abbe486-5915-4204-97ef-54c97b0625f6 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:13.653062 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d8e529-4bfb-4eec-9ab8-5fccec7a43ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:13.653995 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8e529-4bfb-4eec-9ab8-5fccec7a43ca None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:13.654281 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8e529-4bfb-4eec-9ab8-5fccec7a43ca None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:13.654555 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8e529-4bfb-4eec-9ab8-5fccec7a43ca None admin] PATH_INFO: `/v3/users/d2fcecc758494f3aa7df97cae3c0b56d` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:13.656547 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d8e529-4bfb-4eec-9ab8-5fccec7a43ca None admin] RBAC: Authorizing `identity:get_user(user_id=d2fcecc758494f3aa7df97cae3c0b56d)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:13.658005 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d8e529-4bfb-4eec-9ab8-5fccec7a43ca None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:13.659901 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 116/231] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:01:13 2022] GET /identity/v3/users/d2fcecc758494f3aa7df97cae3c0b56d => generated 283 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:13.667087 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d8e529-4bfb-4eec-9ab8-5fccec7a43ca None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:13.697070 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:13.816953 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abbe486-5915-4204-97ef-54c97b0625f6 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:13.837967 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:13.856149 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d8e529-4bfb-4eec-9ab8-5fccec7a43ca None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:13.888784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae5a540-80d2-4373-bbde-5f5ea22fd7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:13.890045 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5a540-80d2-4373-bbde-5f5ea22fd7d5 None admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:13.890542 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5a540-80d2-4373-bbde-5f5ea22fd7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:13.891035 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5a540-80d2-4373-bbde-5f5ea22fd7d5 None admin] PATH_INFO: `/v3/projects/b64d10f5c7424d909a79d57916eaeac5/users/d2fcecc758494f3aa7df97cae3c0b56d/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:13.894492 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1738dca-a219-4342-9baa-1d0072b76760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:13.895357 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1738dca-a219-4342-9baa-1d0072b76760 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:13.895668 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1738dca-a219-4342-9baa-1d0072b76760 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:13.895947 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1738dca-a219-4342-9baa-1d0072b76760 None admin] PATH_INFO: `/v3/projects/b64d10f5c7424d909a79d57916eaeac5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:13.896702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae5a540-80d2-4373-bbde-5f5ea22fd7d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=b64d10f5c7424d909a79d57916eaeac5, user_id=d2fcecc758494f3aa7df97cae3c0b56d, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:13.897149 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1738dca-a219-4342-9baa-1d0072b76760 None admin] RBAC: Authorizing `identity:get_project(project_id=b64d10f5c7424d909a79d57916eaeac5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:13.898600 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1738dca-a219-4342-9baa-1d0072b76760 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:13.898789 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae5a540-80d2-4373-bbde-5f5ea22fd7d5 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:13.905340 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 117/232] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:01:13 2022] GET /identity/v3/projects/b64d10f5c7424d909a79d57916eaeac5 => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:13.909833 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1738dca-a219-4342-9baa-1d0072b76760 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:13.925338 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 116/233] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jul 13 23:01:13 2022] PUT /identity/v3/projects/b64d10f5c7424d909a79d57916eaeac5/users/d2fcecc758494f3aa7df97cae3c0b56d/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:01:14.088516 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1738dca-a219-4342-9baa-1d0072b76760 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:14.131047 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac772a-8960-4bbd-8c4a-d41b04bc3941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:14.131997 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac772a-8960-4bbd-8c4a-d41b04bc3941 None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:14.132279 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac772a-8960-4bbd-8c4a-d41b04bc3941 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:14.132552 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac772a-8960-4bbd-8c4a-d41b04bc3941 None admin] PATH_INFO: `/v3/projects/b64d10f5c7424d909a79d57916eaeac5/users/d2fcecc758494f3aa7df97cae3c0b56d/roles/5f0ca8371a394d339621748b570e78ee` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:14.134912 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcac772a-8960-4bbd-8c4a-d41b04bc3941 None admin] RBAC: Authorizing `identity:create_grant(project_id=b64d10f5c7424d909a79d57916eaeac5, user_id=d2fcecc758494f3aa7df97cae3c0b56d, role_id=5f0ca8371a394d339621748b570e78ee)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:14.136453 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcac772a-8960-4bbd-8c4a-d41b04bc3941 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:14.154004 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 118/234] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jul 13 23:01:13 2022] PUT /identity/v3/projects/b64d10f5c7424d909a79d57916eaeac5/users/d2fcecc758494f3aa7df97cae3c0b56d/roles/5f0ca8371a394d339621748b570e78ee => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:01:14.502049 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc2609c-3c62-4a3e-8231-1095048b5228 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:14.502577 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc2609c-3c62-4a3e-8231-1095048b5228 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:14.502853 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc2609c-3c62-4a3e-8231-1095048b5228 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:14.504590 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 117/235] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:14.510183 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7b938-aca6-4af4-984a-8a951a807833 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:14.510692 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7b938-aca6-4af4-984a-8a951a807833 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:14.511136 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7b938-aca6-4af4-984a-8a951a807833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:14.531678 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-31e7b938-aca6-4af4-984a-8a951a807833 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:14.533191 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-31e7b938-aca6-4af4-984a-8a951a807833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:14.551731 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:14.616105 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 119/236] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:14.640472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc2609c-3c62-4a3e-8231-1095048b5228 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:14.703823 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6d4f5-4450-481d-b280-255b4c15db22 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:14.704454 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6d4f5-4450-481d-b280-255b4c15db22 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:14.704947 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6d4f5-4450-481d-b280-255b4c15db22 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:14.706809 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 120/237] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:01:14.712366 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2f93d-1b61-4e88-9b58-2d0145252a1e None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:14.712954 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2f93d-1b61-4e88-9b58-2d0145252a1e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:14.713421 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2f93d-1b61-4e88-9b58-2d0145252a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:14.733591 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-5be2f93d-1b61-4e88-9b58-2d0145252a1e None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:14.735117 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5be2f93d-1b61-4e88-9b58-2d0145252a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:14.812025 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 121/238] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:14.839104 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2f93d-1b61-4e88-9b58-2d0145252a1e None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:14.857278 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:14.877188 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc2609c-3c62-4a3e-8231-1095048b5228 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:14.924053 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae6bad-3428-4950-973d-8a482477b2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:14.925099 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae6bad-3428-4950-973d-8a482477b2b4 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:14.925545 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae6bad-3428-4950-973d-8a482477b2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:14.925955 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae6bad-3428-4950-973d-8a482477b2b4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:14.934260 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:14.939521 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ae6bad-3428-4950-973d-8a482477b2b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:14.949873 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-01ae6bad-3428-4950-973d-8a482477b2b4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 23:01:14.949873 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:14.949873 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:14.949873 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:14.949873 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:14.949873 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:14.949873 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:14.949873 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:14.949873 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:14.949873 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:14.949873 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:14.949873 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:14.949873 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:14.949873 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:14.949873 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:14.949873 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:14.950870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:14.950870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:14.950870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:14.950870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:14.950870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:14.950870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:14.950870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:14.950870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:14.950870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:14.950870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:14.950870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:14.950870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:14.950870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:14.950870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:14.950870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:14.950870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:14.951735 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:14.951735 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:14.951735 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:14.951735 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:14.951735 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:14.951735 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:14.951735 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:14.951735 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:14.951735 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:14.951735 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:14.951735 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 23:01:14.951735 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:14.954459 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 118/239] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:01:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:14.963563 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2f93d-1b61-4e88-9b58-2d0145252a1e None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:14.964888 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae6bad-3428-4950-973d-8a482477b2b4 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:14.999089 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0368c58-3148-44a1-8494-265789485c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:14.999928 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0368c58-3148-44a1-8494-265789485c2d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:15.000210 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0368c58-3148-44a1-8494-265789485c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:15.000478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0368c58-3148-44a1-8494-265789485c2d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:15.001888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0368c58-3148-44a1-8494-265789485c2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:15.003944 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0368c58-3148-44a1-8494-265789485c2d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:15.005895 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 122/240] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:01:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:15.011828 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0368c58-3148-44a1-8494-265789485c2d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:15.130397 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0368c58-3148-44a1-8494-265789485c2d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:15.172825 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bf6635-ca69-4d1f-885d-daf0414000e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:15.173517 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf6635-ca69-4d1f-885d-daf0414000e9 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:15.174029 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf6635-ca69-4d1f-885d-daf0414000e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:15.174311 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf6635-ca69-4d1f-885d-daf0414000e9 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:15.175022 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bf6635-ca69-4d1f-885d-daf0414000e9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:15.176497 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bf6635-ca69-4d1f-885d-daf0414000e9 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:15.176719 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae6bad-3428-4950-973d-8a482477b2b4 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:15.206418 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 123/241] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Jul 13 23:01:15 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:01:15.257776 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f958263d-a6a6-4da0-b0ba-8eb40e490705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:15.258841 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f958263d-a6a6-4da0-b0ba-8eb40e490705 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:15.259349 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f958263d-a6a6-4da0-b0ba-8eb40e490705 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:15.259892 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f958263d-a6a6-4da0-b0ba-8eb40e490705 None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:15.260876 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f958263d-a6a6-4da0-b0ba-8eb40e490705 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:15.261391 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f958263d-a6a6-4da0-b0ba-8eb40e490705 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:15.263031 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f958263d-a6a6-4da0-b0ba-8eb40e490705 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:15.279327 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 119/242] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jul 13 23:01:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:15.285154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bf6635-ca69-4d1f-885d-daf0414000e9 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:15.288325 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:15.430831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bf6635-ca69-4d1f-885d-daf0414000e9 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:15.469966 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a42c5-2b4b-4413-a5f5-05013812498e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:15.470865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a42c5-2b4b-4413-a5f5-05013812498e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:15.471147 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a42c5-2b4b-4413-a5f5-05013812498e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:15.471421 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a42c5-2b4b-4413-a5f5-05013812498e None admin] PATH_INFO: `/v3/users/7eb8a38ea83b488fae09e62392d9b449` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:15.487208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727a42c5-2b4b-4413-a5f5-05013812498e None admin] RBAC: Authorizing `identity:get_user(user_id=7eb8a38ea83b488fae09e62392d9b449)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:15.488781 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727a42c5-2b4b-4413-a5f5-05013812498e None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:15.490565 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 124/243] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:01:15 2022] GET /identity/v3/users/7eb8a38ea83b488fae09e62392d9b449 => generated 297 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:15.496258 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f958263d-a6a6-4da0-b0ba-8eb40e490705 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:15.524160 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:15.627083 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f958263d-a6a6-4da0-b0ba-8eb40e490705 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:15.668301 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa843ef-6517-48c9-9d84-ed0956039dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:15.669061 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa843ef-6517-48c9-9d84-ed0956039dd7 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:15.669259 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa843ef-6517-48c9-9d84-ed0956039dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:15.669540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa843ef-6517-48c9-9d84-ed0956039dd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:15.673276 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de495d13-ef92-4d9d-90a0-6091a20c9320 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:15.673693 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de495d13-ef92-4d9d-90a0-6091a20c9320 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:15.673901 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de495d13-ef92-4d9d-90a0-6091a20c9320 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:15.675214 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 125/244] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:01:15.680824 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6fc34f-2182-4148-b862-788e403e9cca None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:15.681163 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6fc34f-2182-4148-b862-788e403e9cca None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:15.681389 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6fc34f-2182-4148-b862-788e403e9cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:15.705718 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-ba6fc34f-2182-4148-b862-788e403e9cca None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:15.707417 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ba6fc34f-2182-4148-b862-788e403e9cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:15.755421 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa843ef-6517-48c9-9d84-ed0956039dd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:15.757429 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa843ef-6517-48c9-9d84-ed0956039dd7 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:15.782103 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:15.807337 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:15.835891 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 126/245] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 23:01:15.871127 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6fc34f-2182-4148-b862-788e403e9cca None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:15.889729 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 120/246] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:01:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 23:01:15.895370 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa843ef-6517-48c9-9d84-ed0956039dd7 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:16.018491 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6fc34f-2182-4148-b862-788e403e9cca None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:16.029758 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa843ef-6517-48c9-9d84-ed0956039dd7 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:16.065706 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:16.085130 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4959be5b-94cf-4301-93d1-bec6afa75b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:16.085933 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4959be5b-94cf-4301-93d1-bec6afa75b94 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:16.086206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4959be5b-94cf-4301-93d1-bec6afa75b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:16.086477 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4959be5b-94cf-4301-93d1-bec6afa75b94 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:16.096633 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273fc49-46cf-45fb-ad20-4cd52fe063eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:16.097497 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273fc49-46cf-45fb-ad20-4cd52fe063eb None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:16.097775 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273fc49-46cf-45fb-ad20-4cd52fe063eb None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:16.098045 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273fc49-46cf-45fb-ad20-4cd52fe063eb None admin] PATH_INFO: `/v3/projects/b64d10f5c7424d909a79d57916eaeac5` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:16.099456 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f273fc49-46cf-45fb-ad20-4cd52fe063eb None admin] RBAC: Authorizing `identity:get_project(project_id=b64d10f5c7424d909a79d57916eaeac5)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:16.100976 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f273fc49-46cf-45fb-ad20-4cd52fe063eb None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:16.102731 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 121/247] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:01:15 2022] GET /identity/v3/projects/b64d10f5c7424d909a79d57916eaeac5 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:16.108727 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273fc49-46cf-45fb-ad20-4cd52fe063eb None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:16.124513 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4959be5b-94cf-4301-93d1-bec6afa75b94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:16.132717 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-4959be5b-94cf-4301-93d1-bec6afa75b94 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 23:01:16.132717 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:16.132717 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:16.132717 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:16.132717 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:16.132717 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:16.132717 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:16.132717 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:16.132717 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:16.132717 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:16.132717 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:16.132717 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:16.132717 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:16.132717 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:16.132717 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:16.132717 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:16.133831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:16.133831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:16.133831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:16.133831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:16.133831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:16.133831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:16.133831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:16.133831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:16.133831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:16.133831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:16.133831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:16.133831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:16.133831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:16.133831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:16.133831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:16.133831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:16.140047 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:16.140047 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:16.140047 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:16.140047 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:16.140047 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:16.140047 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:16.140047 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:16.140047 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:16.140047 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:16.140047 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:16.140047 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 23:01:16.140047 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:16.143144 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 127/248] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:01:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:01:16.157253 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4959be5b-94cf-4301-93d1-bec6afa75b94 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:16.236442 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273fc49-46cf-45fb-ad20-4cd52fe063eb None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:16.275749 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936d416-0b99-4182-9f1b-9c89041d5977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:16.276729 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936d416-0b99-4182-9f1b-9c89041d5977 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:16.277147 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936d416-0b99-4182-9f1b-9c89041d5977 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:16.277568 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936d416-0b99-4182-9f1b-9c89041d5977 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:16.278984 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c936d416-0b99-4182-9f1b-9c89041d5977 None admin] RBAC: Adding query filter params (role.id=2562dfc9fd0845678cb5beff6a715255, scope.project.id=b64d10f5c7424d909a79d57916eaeac5, user.id=7eb8a38ea83b488fae09e62392d9b449) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:16.279448 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c936d416-0b99-4182-9f1b-9c89041d5977 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:16.281071 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c936d416-0b99-4182-9f1b-9c89041d5977 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:16.292719 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 122/249] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:01:16 2022] GET /identity/v3/role_assignments?user.id=7eb8a38ea83b488fae09e62392d9b449&scope.project.id=b64d10f5c7424d909a79d57916eaeac5&role.id=2562dfc9fd0845678cb5beff6a715255 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:01:16.323997 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:16.442289 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4959be5b-94cf-4301-93d1-bec6afa75b94 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:16.483993 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c5f05-2f10-4c35-8736-deb69e13361d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:16.484986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c5f05-2f10-4c35-8736-deb69e13361d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:16.485443 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c5f05-2f10-4c35-8736-deb69e13361d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:16.485889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c5f05-2f10-4c35-8736-deb69e13361d None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:16.486819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226c5f05-2f10-4c35-8736-deb69e13361d None admin] RBAC: Adding query filter params (name=member) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:16.487326 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226c5f05-2f10-4c35-8736-deb69e13361d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:16.488962 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226c5f05-2f10-4c35-8736-deb69e13361d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:16.500197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 128/250] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jul 13 23:01:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:01:16.506338 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936d416-0b99-4182-9f1b-9c89041d5977 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:16.565951 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:16.577302 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377a146-5e80-49bb-b32f-f8cce20d13b2 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:16.577908 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377a146-5e80-49bb-b32f-f8cce20d13b2 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:16.578357 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377a146-5e80-49bb-b32f-f8cce20d13b2 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:16.580007 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 129/251] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:01:16.590330 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78297e-b808-4623-8229-6609ff612738 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:16.590892 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78297e-b808-4623-8229-6609ff612738 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:16.591365 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78297e-b808-4623-8229-6609ff612738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:16.615312 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-cc78297e-b808-4623-8229-6609ff612738 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:16.616912 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-cc78297e-b808-4623-8229-6609ff612738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:16.664035 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936d416-0b99-4182-9f1b-9c89041d5977 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:16.702848 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 130/252] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 23:01:16.720957 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e9c5f8-a78c-48de-809a-0e7d33442bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:16.721857 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9c5f8-a78c-48de-809a-0e7d33442bcf None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:16.722143 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9c5f8-a78c-48de-809a-0e7d33442bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:16.722423 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9c5f8-a78c-48de-809a-0e7d33442bcf None admin] PATH_INFO: `/v3/users/ea11955c44dd41cc998c143ef68f43d2` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:16.728942 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e9c5f8-a78c-48de-809a-0e7d33442bcf None admin] RBAC: Authorizing `identity:get_user(user_id=ea11955c44dd41cc998c143ef68f43d2)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:16.730449 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e9c5f8-a78c-48de-809a-0e7d33442bcf None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:16.732564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 123/253] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:01:16 2022] GET /identity/v3/users/ea11955c44dd41cc998c143ef68f43d2 => generated 297 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:16.738198 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc78297e-b808-4623-8229-6609ff612738 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:16.750096 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e9c5f8-a78c-48de-809a-0e7d33442bcf None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:16.816112 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:16.912670 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc78297e-b808-4623-8229-6609ff612738 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:16.966400 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef7d30-48f0-477d-a046-264c8f3b6b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:16.967296 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef7d30-48f0-477d-a046-264c8f3b6b3f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:16.967618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef7d30-48f0-477d-a046-264c8f3b6b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:16.967901 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef7d30-48f0-477d-a046-264c8f3b6b3f None admin] PATH_INFO: `/v3/projects/b64d10f5c7424d909a79d57916eaeac5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:16.969152 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef7d30-48f0-477d-a046-264c8f3b6b3f None admin] RBAC: Authorizing `identity:get_project(project_id=b64d10f5c7424d909a79d57916eaeac5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:16.970631 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef7d30-48f0-477d-a046-264c8f3b6b3f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:16.972754 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 131/254] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:01:16 2022] GET /identity/v3/projects/b64d10f5c7424d909a79d57916eaeac5 => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:16.974202 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc36af9-e1ae-4712-80ab-ec2be69948e1 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:16.974490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc36af9-e1ae-4712-80ab-ec2be69948e1 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:16.974769 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc36af9-e1ae-4712-80ab-ec2be69948e1 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:16.981723 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 132/255] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:16.982660 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc36af9-e1ae-4712-80ab-ec2be69948e1 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:17.086982 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e9c5f8-a78c-48de-809a-0e7d33442bcf None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:17.127915 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:17.151863 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc36af9-e1ae-4712-80ab-ec2be69948e1 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:17.162623 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f27c9a-f4d4-4e8a-b9d1-dc68137d44fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:17.163426 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f27c9a-f4d4-4e8a-b9d1-dc68137d44fd None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:17.163771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f27c9a-f4d4-4e8a-b9d1-dc68137d44fd None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:17.164045 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f27c9a-f4d4-4e8a-b9d1-dc68137d44fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:17.172773 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:17.183099 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f27c9a-f4d4-4e8a-b9d1-dc68137d44fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:17.202721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-18f27c9a-f4d4-4e8a-b9d1-dc68137d44fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:01:17.202721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:17.202721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:17.202721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:17.202721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:17.202721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:17.202721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:17.202721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:17.202721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:17.202721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:17.202721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:17.202721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:17.202721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:17.202721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:17.202721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:17.202721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:17.203623 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:17.203623 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:17.203623 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:17.203623 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:17.203623 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:17.203623 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:17.203623 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:17.203623 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:17.203623 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:17.203623 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:17.203623 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:17.203623 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:17.203623 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:17.203623 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:17.203623 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:17.203623 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:17.204403 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:17.204403 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:17.204403 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:17.204403 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:17.204403 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:17.204403 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:17.204403 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:17.204403 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:17.204403 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:17.204403 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:17.204403 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:01:17.204403 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:17.204403 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 124/256] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jul 13 23:01:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 458 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:01:17.205792 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e29c52-f7f9-43fd-ac12-f5a18c63b4ce None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:17.209694 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e29c52-f7f9-43fd-ac12-f5a18c63b4ce None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:17.210161 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e29c52-f7f9-43fd-ac12-f5a18c63b4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:17.221569 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ebdf4-90c9-4387-874a-40583fe296da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:17.222465 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ebdf4-90c9-4387-874a-40583fe296da None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:17.222739 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ebdf4-90c9-4387-874a-40583fe296da None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:17.223005 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ebdf4-90c9-4387-874a-40583fe296da None admin] PATH_INFO: `/v3/projects/b64d10f5c7424d909a79d57916eaeac5/users/ea11955c44dd41cc998c143ef68f43d2/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:17.231398 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72ebdf4-90c9-4387-874a-40583fe296da None admin] RBAC: Authorizing `identity:create_grant(project_id=b64d10f5c7424d909a79d57916eaeac5, user_id=ea11955c44dd41cc998c143ef68f43d2, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:17.233185 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72ebdf4-90c9-4387-874a-40583fe296da None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:17.240714 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-15e29c52-f7f9-43fd-ac12-f5a18c63b4ce None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:17.242430 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-15e29c52-f7f9-43fd-ac12-f5a18c63b4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:17.264742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 133/257] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jul 13 23:01:16 2022] PUT /identity/v3/projects/b64d10f5c7424d909a79d57916eaeac5/users/ea11955c44dd41cc998c143ef68f43d2/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:01:17.271565 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ebdf4-90c9-4387-874a-40583fe296da None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:17.356467 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 125/258] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:17.401034 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e29c52-f7f9-43fd-ac12-f5a18c63b4ce None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:17.524051 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ebdf4-90c9-4387-874a-40583fe296da None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:17.601751 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73875bc-a266-4b63-bbaf-e6b8f90e079d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:17.602426 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73875bc-a266-4b63-bbaf-e6b8f90e079d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:17.602704 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73875bc-a266-4b63-bbaf-e6b8f90e079d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:17.602975 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73875bc-a266-4b63-bbaf-e6b8f90e079d None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:17.604335 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73875bc-a266-4b63-bbaf-e6b8f90e079d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:17.604641 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73875bc-a266-4b63-bbaf-e6b8f90e079d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:17.606043 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73875bc-a266-4b63-bbaf-e6b8f90e079d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:17.608337 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e29c52-f7f9-43fd-ac12-f5a18c63b4ce None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:17.631865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 134/259] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:01:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:17.632563 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:17.637702 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73875bc-a266-4b63-bbaf-e6b8f90e079d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:17.688317 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3727f2-c236-4902-b2b1-20290eceb2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:17.689132 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3727f2-c236-4902-b2b1-20290eceb2d9 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:17.689410 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3727f2-c236-4902-b2b1-20290eceb2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:17.689679 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3727f2-c236-4902-b2b1-20290eceb2d9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:17.699935 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:17.716246 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3727f2-c236-4902-b2b1-20290eceb2d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:17.733518 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-3b3727f2-c236-4902-b2b1-20290eceb2d9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 23:01:17.733518 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:17.733518 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:17.733518 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:17.733518 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:17.733518 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:17.733518 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:17.733518 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:17.733518 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:17.733518 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:17.733518 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:17.733518 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:17.733518 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:17.733518 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:17.733518 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:17.733518 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:17.734877 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:17.734877 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:17.734877 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:17.734877 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:17.734877 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:17.734877 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:17.734877 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:17.734877 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:17.734877 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:17.734877 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:17.734877 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:17.734877 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:17.734877 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:17.734877 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:17.734877 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:17.734877 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:17.735705 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:17.735705 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:17.735705 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:17.735705 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:17.735705 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:17.735705 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:17.735705 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:17.735705 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:17.735705 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:17.735705 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:17.735705 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 23:01:17.735705 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:17.735705 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 126/260] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jul 13 23:01:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:01:17.753731 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3727f2-c236-4902-b2b1-20290eceb2d9 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:17.852253 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73875bc-a266-4b63-bbaf-e6b8f90e079d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:17.922073 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1b3c0-79fd-4e96-a539-189af52838aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:17.924338 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1b3c0-79fd-4e96-a539-189af52838aa None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:17.925016 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1b3c0-79fd-4e96-a539-189af52838aa None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:17.926633 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1b3c0-79fd-4e96-a539-189af52838aa None admin] PATH_INFO: `/v3/users/d2fcecc758494f3aa7df97cae3c0b56d` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:17.928294 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f1b3c0-79fd-4e96-a539-189af52838aa None admin] RBAC: Authorizing `identity:get_user(user_id=d2fcecc758494f3aa7df97cae3c0b56d)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:17.930888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f1b3c0-79fd-4e96-a539-189af52838aa None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:17.936403 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 135/261] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:01:17 2022] GET /identity/v3/users/d2fcecc758494f3aa7df97cae3c0b56d => generated 283 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:17.941290 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1b3c0-79fd-4e96-a539-189af52838aa None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:17.965192 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3727f2-c236-4902-b2b1-20290eceb2d9 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:18.044700 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa0904-2004-4f67-bf43-8c827aeaa5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:18.045732 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa0904-2004-4f67-bf43-8c827aeaa5e4 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:18.046384 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa0904-2004-4f67-bf43-8c827aeaa5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:18.047966 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa0904-2004-4f67-bf43-8c827aeaa5e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:18.048769 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa0904-2004-4f67-bf43-8c827aeaa5e4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:18.049090 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa0904-2004-4f67-bf43-8c827aeaa5e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:18.050525 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aa0904-2004-4f67-bf43-8c827aeaa5e4 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:18.066066 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 127/262] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:01:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:18.075349 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa0904-2004-4f67-bf43-8c827aeaa5e4 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:18.146584 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:18.161950 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:18.167810 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1b3c0-79fd-4e96-a539-189af52838aa None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:18.206877 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d365bb-ed9c-48a7-9b8b-42a215b48a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:18.207932 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d365bb-ed9c-48a7-9b8b-42a215b48a60 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:18.208476 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d365bb-ed9c-48a7-9b8b-42a215b48a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:18.208977 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d365bb-ed9c-48a7-9b8b-42a215b48a60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:18.408243 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa0904-2004-4f67-bf43-8c827aeaa5e4 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:18.468924 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d365bb-ed9c-48a7-9b8b-42a215b48a60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:18.471506 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d365bb-ed9c-48a7-9b8b-42a215b48a60 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:18.532886 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accf7e2f-3b6a-4e56-999f-b0e0e7776311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:18.534244 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf7e2f-3b6a-4e56-999f-b0e0e7776311 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:18.534526 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf7e2f-3b6a-4e56-999f-b0e0e7776311 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:18.534801 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf7e2f-3b6a-4e56-999f-b0e0e7776311 None admin] PATH_INFO: `/v3/users/d2fcecc758494f3aa7df97cae3c0b56d` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:18.540359 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accf7e2f-3b6a-4e56-999f-b0e0e7776311 None admin] RBAC: Authorizing `identity:get_user(user_id=d2fcecc758494f3aa7df97cae3c0b56d)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:18.542040 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accf7e2f-3b6a-4e56-999f-b0e0e7776311 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:18.544122 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 128/263] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:01:18 2022] GET /identity/v3/users/d2fcecc758494f3aa7df97cae3c0b56d => generated 283 bytes in 469 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:18.549895 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accf7e2f-3b6a-4e56-999f-b0e0e7776311 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:18.871656 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:18.881940 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 136/264] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:01:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 941 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 13 23:01:18.884330 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae88ae0-1b79-48a8-9dac-74693b2b5552 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:18.895714 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae88ae0-1b79-48a8-9dac-74693b2b5552 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:18.896446 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae88ae0-1b79-48a8-9dac-74693b2b5552 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:18.899456 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 137/265] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:18 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:18.900479 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae88ae0-1b79-48a8-9dac-74693b2b5552 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:18.936083 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:18.940266 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accf7e2f-3b6a-4e56-999f-b0e0e7776311 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:18.980764 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf55b13e-4b8e-4755-affb-ea5d5db6465a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:18.981855 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55b13e-4b8e-4755-affb-ea5d5db6465a None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:18.982669 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55b13e-4b8e-4755-affb-ea5d5db6465a None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:18.983163 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55b13e-4b8e-4755-affb-ea5d5db6465a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:19.019226 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae88ae0-1b79-48a8-9dac-74693b2b5552 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:19.054363 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7fd4a-4ea2-412a-b475-408425d13c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:19.055217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7fd4a-4ea2-412a-b475-408425d13c29 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:19.055491 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7fd4a-4ea2-412a-b475-408425d13c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:19.055784 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7fd4a-4ea2-412a-b475-408425d13c29 None admin] PATH_INFO: `/v3/projects/b64d10f5c7424d909a79d57916eaeac5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:19.057034 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef7fd4a-4ea2-412a-b475-408425d13c29 None admin] RBAC: Authorizing `identity:get_project(project_id=b64d10f5c7424d909a79d57916eaeac5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:19.058533 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef7fd4a-4ea2-412a-b475-408425d13c29 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:19.060680 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 138/266] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:01:18 2022] GET /identity/v3/projects/b64d10f5c7424d909a79d57916eaeac5 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:19.061974 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421229dc-4654-493b-bfb5-7c745c94e42c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:19.062330 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421229dc-4654-493b-bfb5-7c745c94e42c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:19.062606 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421229dc-4654-493b-bfb5-7c745c94e42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:19.087382 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-421229dc-4654-493b-bfb5-7c745c94e42c None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:19.089156 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-421229dc-4654-493b-bfb5-7c745c94e42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:19.096030 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf55b13e-4b8e-4755-affb-ea5d5db6465a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:19.097503 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf55b13e-4b8e-4755-affb-ea5d5db6465a None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:19.155487 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:19.163328 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 139/267] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:19.164615 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c39ed0-25e0-4eba-b72f-debce70db3b5 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:19.164899 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c39ed0-25e0-4eba-b72f-debce70db3b5 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:19.165173 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c39ed0-25e0-4eba-b72f-debce70db3b5 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:19.171036 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 140/268] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:19.171036 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc8965a-f24f-4076-b97d-f85cabdd3b23 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:19.171036 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc8965a-f24f-4076-b97d-f85cabdd3b23 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:19.171036 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc8965a-f24f-4076-b97d-f85cabdd3b23 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:19.171036 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 141/269] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:19.171036 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc8965a-f24f-4076-b97d-f85cabdd3b23 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:19.206479 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:19.210973 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 129/270] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:01:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 662 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 23:01:19.213023 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fda02c-4aca-4411-b019-c6d654fa7d70 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:19.213992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fda02c-4aca-4411-b019-c6d654fa7d70 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:19.214273 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fda02c-4aca-4411-b019-c6d654fa7d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:19.232122 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-58fda02c-4aca-4411-b019-c6d654fa7d70 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:19.233313 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-58fda02c-4aca-4411-b019-c6d654fa7d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:19.294823 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc8965a-f24f-4076-b97d-f85cabdd3b23 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:19.307834 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 130/271] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:19.309098 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052d0d4-241e-46d8-9a2f-2203b9844ff7 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:19.309577 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052d0d4-241e-46d8-9a2f-2203b9844ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:19.309926 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052d0d4-241e-46d8-9a2f-2203b9844ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:19.326997 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-0052d0d4-241e-46d8-9a2f-2203b9844ff7 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:19.328391 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-0052d0d4-241e-46d8-9a2f-2203b9844ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:19.353915 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d39b0-f5b2-441d-a424-4905d2206739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:19.355587 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d39b0-f5b2-441d-a424-4905d2206739 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:19.355587 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d39b0-f5b2-441d-a424-4905d2206739 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:19.355587 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d39b0-f5b2-441d-a424-4905d2206739 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:19.360057 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3d39b0-f5b2-441d-a424-4905d2206739 None admin] RBAC: Adding query filter params (role.id=8cfd0b6e71e14e0983c3eb4d7478e027, scope.project.id=b64d10f5c7424d909a79d57916eaeac5, user.id=d2fcecc758494f3aa7df97cae3c0b56d) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:19.360057 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3d39b0-f5b2-441d-a424-4905d2206739 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:19.360057 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3d39b0-f5b2-441d-a424-4905d2206739 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:19.365442 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 142/272] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:01:19 2022] GET /identity/v3/role_assignments?user.id=d2fcecc758494f3aa7df97cae3c0b56d&scope.project.id=b64d10f5c7424d909a79d57916eaeac5&role.id=8cfd0b6e71e14e0983c3eb4d7478e027 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:19.366108 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d39b0-f5b2-441d-a424-4905d2206739 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:19.417785 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 131/273] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:19.419175 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0052d0d4-241e-46d8-9a2f-2203b9844ff7 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:19.458141 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:19.503648 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:19.510773 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d39b0-f5b2-441d-a424-4905d2206739 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:19.551553 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0052d0d4-241e-46d8-9a2f-2203b9844ff7 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:19.551742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f952debc-6e0c-4d97-810c-cbfd1eb89741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:19.552799 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952debc-6e0c-4d97-810c-cbfd1eb89741 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:19.553188 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952debc-6e0c-4d97-810c-cbfd1eb89741 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:19.553470 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952debc-6e0c-4d97-810c-cbfd1eb89741 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:19.567312 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f952debc-6e0c-4d97-810c-cbfd1eb89741 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:19.576033 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-f952debc-6e0c-4d97-810c-cbfd1eb89741 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 23:01:19.576033 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:19.576033 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:19.576033 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:19.576033 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:19.576033 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:19.576033 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:19.576033 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:19.576033 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:19.576033 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:19.576033 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:19.576033 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:19.576033 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:19.576033 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:19.576033 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:19.576033 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:19.576987 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:19.576987 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:19.576987 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:19.576987 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:19.576987 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:19.576987 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:19.576987 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:19.576987 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:19.576987 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:19.576987 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:19.576987 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:19.576987 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:19.576987 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:19.576987 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:19.576987 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:19.576987 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:19.577765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:19.577765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:19.577765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:19.577765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:19.577765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:19.577765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:19.577765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:19.577765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:19.577765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:19.577765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:19.577765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 23:01:19.577765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:19.582723 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 143/274] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:01:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:19.583670 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f952debc-6e0c-4d97-810c-cbfd1eb89741 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:19.590385 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94591218-0483-4cde-bb2a-90c73d4456b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:19.591512 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94591218-0483-4cde-bb2a-90c73d4456b0 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:19.591920 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94591218-0483-4cde-bb2a-90c73d4456b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:19.592340 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94591218-0483-4cde-bb2a-90c73d4456b0 None admin] PATH_INFO: `/v3/projects/b64d10f5c7424d909a79d57916eaeac5` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:19.594163 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94591218-0483-4cde-bb2a-90c73d4456b0 None admin] RBAC: Authorizing `identity:get_project(project_id=b64d10f5c7424d909a79d57916eaeac5)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:19.596179 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94591218-0483-4cde-bb2a-90c73d4456b0 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:19.598367 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 132/275] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:01:19 2022] GET /identity/v3/projects/b64d10f5c7424d909a79d57916eaeac5 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:19.599149 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94591218-0483-4cde-bb2a-90c73d4456b0 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:19.709514 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f952debc-6e0c-4d97-810c-cbfd1eb89741 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:19.713859 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94591218-0483-4cde-bb2a-90c73d4456b0 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:19.736101 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:19.746513 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:19.751130 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc2c00d-734d-4b10-8f0b-dd9b2e22f98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:19.751984 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2c00d-734d-4b10-8f0b-dd9b2e22f98f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:19.752272 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2c00d-734d-4b10-8f0b-dd9b2e22f98f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:19.752551 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2c00d-734d-4b10-8f0b-dd9b2e22f98f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:19.757461 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1cd38a-e9a4-4461-ace8-c1c7dc1a6f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:19.758871 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1cd38a-e9a4-4461-ace8-c1c7dc1a6f0f None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:19.759340 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1cd38a-e9a4-4461-ace8-c1c7dc1a6f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:19.759832 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1cd38a-e9a4-4461-ace8-c1c7dc1a6f0f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:19.762004 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1cd38a-e9a4-4461-ace8-c1c7dc1a6f0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:19.764125 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1cd38a-e9a4-4461-ace8-c1c7dc1a6f0f None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:19.764353 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc2c00d-734d-4b10-8f0b-dd9b2e22f98f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:19.766502 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 133/276] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:01:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:19.767441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1cd38a-e9a4-4461-ace8-c1c7dc1a6f0f None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:19.772580 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-adc2c00d-734d-4b10-8f0b-dd9b2e22f98f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 23:01:19.772580 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:19.772580 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:19.772580 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:19.772580 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:19.772580 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:19.772580 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:19.772580 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:19.772580 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:19.772580 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:19.772580 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:19.772580 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:19.772580 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:19.772580 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:19.772580 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:19.772580 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:19.773680 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:19.773680 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:19.773680 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:19.773680 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:19.773680 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:19.773680 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:19.773680 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:19.773680 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:19.773680 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:19.773680 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:19.773680 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:19.773680 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:19.773680 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:19.773680 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:19.773680 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:19.773680 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:19.774516 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:19.774516 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:19.774516 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:19.774516 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:19.774516 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:19.774516 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:19.774516 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:19.774516 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:19.774516 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:19.774516 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:19.774516 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 23:01:19.774516 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:19.774516 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 144/277] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:01:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:19.775292 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc2c00d-734d-4b10-8f0b-dd9b2e22f98f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:19.886903 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc2c00d-734d-4b10-8f0b-dd9b2e22f98f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:19.895655 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1cd38a-e9a4-4461-ace8-c1c7dc1a6f0f None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:19.922973 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2267cd8-c2d9-4f75-a866-5a637d3ef584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:19.923764 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2267cd8-c2d9-4f75-a866-5a637d3ef584 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:19.924047 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2267cd8-c2d9-4f75-a866-5a637d3ef584 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:19.924330 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2267cd8-c2d9-4f75-a866-5a637d3ef584 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:19.925171 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2267cd8-c2d9-4f75-a866-5a637d3ef584 None admin] RBAC: Adding query filter params (role.id=5f0ca8371a394d339621748b570e78ee, scope.project.id=b64d10f5c7424d909a79d57916eaeac5, user.id=d2fcecc758494f3aa7df97cae3c0b56d) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:19.925483 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2267cd8-c2d9-4f75-a866-5a637d3ef584 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:19.926948 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2267cd8-c2d9-4f75-a866-5a637d3ef584 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:19.933169 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 145/278] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:01:19 2022] GET /identity/v3/role_assignments?user.id=d2fcecc758494f3aa7df97cae3c0b56d&scope.project.id=b64d10f5c7424d909a79d57916eaeac5&role.id=5f0ca8371a394d339621748b570e78ee => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:19.933963 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2267cd8-c2d9-4f75-a866-5a637d3ef584 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:19.935981 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde3d0a-cbc6-49a5-9f0d-bb29732a7bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:19.936875 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde3d0a-cbc6-49a5-9f0d-bb29732a7bbc None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:19.937455 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde3d0a-cbc6-49a5-9f0d-bb29732a7bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:19.937455 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde3d0a-cbc6-49a5-9f0d-bb29732a7bbc None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:19.938231 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdde3d0a-cbc6-49a5-9f0d-bb29732a7bbc None admin] RBAC: Adding query filter params (name=member) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:19.938541 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdde3d0a-cbc6-49a5-9f0d-bb29732a7bbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:19.940031 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdde3d0a-cbc6-49a5-9f0d-bb29732a7bbc None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:19.959502 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 134/279] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jul 13 23:01:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:19.959502 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde3d0a-cbc6-49a5-9f0d-bb29732a7bbc None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:20.008071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:20.048366 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:20.096820 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde3d0a-cbc6-49a5-9f0d-bb29732a7bbc None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:20.110407 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2267cd8-c2d9-4f75-a866-5a637d3ef584 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:20.138207 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951362bb-ffdb-4191-bf49-add54d343c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:20.138886 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951362bb-ffdb-4191-bf49-add54d343c16 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:20.139174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951362bb-ffdb-4191-bf49-add54d343c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:20.139451 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951362bb-ffdb-4191-bf49-add54d343c16 None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:20.140403 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951362bb-ffdb-4191-bf49-add54d343c16 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:20.140712 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951362bb-ffdb-4191-bf49-add54d343c16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:20.142124 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951362bb-ffdb-4191-bf49-add54d343c16 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:20.146716 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad84a00-5a52-43a9-9826-8188fbc847fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:20.147388 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad84a00-5a52-43a9-9826-8188fbc847fc None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:20.147710 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad84a00-5a52-43a9-9826-8188fbc847fc None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:20.147982 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad84a00-5a52-43a9-9826-8188fbc847fc None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:20.148691 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad84a00-5a52-43a9-9826-8188fbc847fc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:20.150676 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad84a00-5a52-43a9-9826-8188fbc847fc None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:20.153506 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 135/280] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jul 13 23:01:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:20.154182 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951362bb-ffdb-4191-bf49-add54d343c16 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:20.174269 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 146/281] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Jul 13 23:01:19 2022] POST /identity/v3/users => generated 293 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:01:20.176941 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2033b405-93d4-4032-8d9b-4eed6fc2a841 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:20.177255 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2033b405-93d4-4032-8d9b-4eed6fc2a841 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:20.177510 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2033b405-93d4-4032-8d9b-4eed6fc2a841 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:20.179123 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 147/282] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:20.179761 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2033b405-93d4-4032-8d9b-4eed6fc2a841 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:20.287419 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951362bb-ffdb-4191-bf49-add54d343c16 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:20.309573 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:20.317015 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2033b405-93d4-4032-8d9b-4eed6fc2a841 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:20.318767 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:20.327876 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df69983-a9cd-4ffd-a32e-9bd4e22528ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:20.328773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df69983-a9cd-4ffd-a32e-9bd4e22528ea None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:20.329100 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df69983-a9cd-4ffd-a32e-9bd4e22528ea None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:20.329375 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df69983-a9cd-4ffd-a32e-9bd4e22528ea None admin] PATH_INFO: `/v3/users/5e56551e804645dd906ee799dd69b727` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:20.345314 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df69983-a9cd-4ffd-a32e-9bd4e22528ea None admin] RBAC: Authorizing `identity:get_user(user_id=5e56551e804645dd906ee799dd69b727)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:20.346802 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df69983-a9cd-4ffd-a32e-9bd4e22528ea None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:20.349024 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 136/283] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:01:20 2022] GET /identity/v3/users/5e56551e804645dd906ee799dd69b727 => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:20.349880 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e0168-4773-47be-af52-07f167c3b569 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:20.350607 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e0168-4773-47be-af52-07f167c3b569 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:20.350607 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e0168-4773-47be-af52-07f167c3b569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:20.371143 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-098e0168-4773-47be-af52-07f167c3b569 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:20.372251 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-098e0168-4773-47be-af52-07f167c3b569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:20.374060 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f028cfb-ed6a-414a-aee2-f1fcba7a2f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:20.374549 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f028cfb-ed6a-414a-aee2-f1fcba7a2f6a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:20.374823 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f028cfb-ed6a-414a-aee2-f1fcba7a2f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:20.375089 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f028cfb-ed6a-414a-aee2-f1fcba7a2f6a None admin] PATH_INFO: `/v3/users/7eb8a38ea83b488fae09e62392d9b449` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:20.376388 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f028cfb-ed6a-414a-aee2-f1fcba7a2f6a None admin] RBAC: Authorizing `identity:get_user(user_id=7eb8a38ea83b488fae09e62392d9b449)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:20.377881 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f028cfb-ed6a-414a-aee2-f1fcba7a2f6a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:20.380212 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 148/284] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:01:20 2022] GET /identity/v3/users/7eb8a38ea83b488fae09e62392d9b449 => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:20.380434 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f028cfb-ed6a-414a-aee2-f1fcba7a2f6a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:20.454641 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 137/285] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:20.455515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e0168-4773-47be-af52-07f167c3b569 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:20.518631 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f028cfb-ed6a-414a-aee2-f1fcba7a2f6a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:20.573311 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677967c7-54eb-4e3e-94cf-2ede9ac7b940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:20.573944 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677967c7-54eb-4e3e-94cf-2ede9ac7b940 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:20.574230 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677967c7-54eb-4e3e-94cf-2ede9ac7b940 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:20.574499 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677967c7-54eb-4e3e-94cf-2ede9ac7b940 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:20.575956 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677967c7-54eb-4e3e-94cf-2ede9ac7b940 None admin] RBAC: Adding query filter params (role.id=f005bdf39a0743238b3476f6b88b3684, user.id=5e56551e804645dd906ee799dd69b727, scope.system=all) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:20.576264 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677967c7-54eb-4e3e-94cf-2ede9ac7b940 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:20.578067 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677967c7-54eb-4e3e-94cf-2ede9ac7b940 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:20.585710 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 149/286] 10.222.0.60 () {56 vars in 1308 bytes} [Wed Jul 13 23:01:20 2022] GET /identity/v3/role_assignments?user.id=5e56551e804645dd906ee799dd69b727&scope.system=all&role.id=f005bdf39a0743238b3476f6b88b3684 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:20.586787 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677967c7-54eb-4e3e-94cf-2ede9ac7b940 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:20.589989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e0168-4773-47be-af52-07f167c3b569 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:20.602118 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:20.639636 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be127a-8301-4ced-96d5-dfb3cf210a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:20.641023 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be127a-8301-4ced-96d5-dfb3cf210a0a None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:20.641502 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be127a-8301-4ced-96d5-dfb3cf210a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:20.641959 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be127a-8301-4ced-96d5-dfb3cf210a0a None admin] PATH_INFO: `/v3/projects/b64d10f5c7424d909a79d57916eaeac5` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:20.643580 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be127a-8301-4ced-96d5-dfb3cf210a0a None admin] RBAC: Authorizing `identity:get_project(project_id=b64d10f5c7424d909a79d57916eaeac5)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:20.645768 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be127a-8301-4ced-96d5-dfb3cf210a0a None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:20.648500 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 138/287] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:01:20 2022] GET /identity/v3/projects/b64d10f5c7424d909a79d57916eaeac5 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:20.657241 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be127a-8301-4ced-96d5-dfb3cf210a0a None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:20.667203 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:20.745286 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677967c7-54eb-4e3e-94cf-2ede9ac7b940 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:20.791489 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798ab8c9-69d3-4ad3-972c-b8b2827deaea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:20.792319 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ab8c9-69d3-4ad3-972c-b8b2827deaea None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:20.792595 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ab8c9-69d3-4ad3-972c-b8b2827deaea None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:20.792868 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ab8c9-69d3-4ad3-972c-b8b2827deaea None admin] PATH_INFO: `/v3/roles/member` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:20.807046 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798ab8c9-69d3-4ad3-972c-b8b2827deaea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:20.814914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-798ab8c9-69d3-4ad3-972c-b8b2827deaea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 23:01:20.814914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:20.814914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:20.814914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:20.814914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:20.814914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:20.814914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:20.814914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:20.814914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:20.814914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:20.814914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:20.814914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:20.814914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:20.814914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:20.814914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:20.814914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:20.815880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:20.815880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:20.815880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:20.815880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:20.815880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:20.815880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:20.815880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:20.815880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:20.815880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:20.815880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:20.815880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:20.815880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:20.815880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:20.815880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:20.815880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:20.815880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:20.816658 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:20.816658 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:20.816658 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:20.816658 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:20.816658 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:20.816658 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:20.816658 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:20.816658 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:20.816658 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:20.816658 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:20.816658 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 23:01:20.816658 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:20.816658 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 150/288] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:01:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:20.824821 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798ab8c9-69d3-4ad3-972c-b8b2827deaea None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:20.862334 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be127a-8301-4ced-96d5-dfb3cf210a0a None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:20.958202 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:20.961202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ca098-d384-4c26-bc77-2c3cd7f3b1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:20.962116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ca098-d384-4c26-bc77-2c3cd7f3b1f4 None admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:20.962399 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ca098-d384-4c26-bc77-2c3cd7f3b1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:20.962742 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ca098-d384-4c26-bc77-2c3cd7f3b1f4 None admin] PATH_INFO: `/v3/projects/b64d10f5c7424d909a79d57916eaeac5/users/7eb8a38ea83b488fae09e62392d9b449/roles/2562dfc9fd0845678cb5beff6a715255` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:20.970040 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798ab8c9-69d3-4ad3-972c-b8b2827deaea None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:20.972865 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6ca098-d384-4c26-bc77-2c3cd7f3b1f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=b64d10f5c7424d909a79d57916eaeac5, user_id=7eb8a38ea83b488fae09e62392d9b449, role_id=2562dfc9fd0845678cb5beff6a715255)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:20.974365 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6ca098-d384-4c26-bc77-2c3cd7f3b1f4 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:21.002315 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 139/289] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jul 13 23:01:20 2022] PUT /identity/v3/projects/b64d10f5c7424d909a79d57916eaeac5/users/7eb8a38ea83b488fae09e62392d9b449/roles/2562dfc9fd0845678cb5beff6a715255 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:01:21.035046 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd9e325-cde4-4533-8a32-5694ff66d94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:21.035775 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9e325-cde4-4533-8a32-5694ff66d94d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:21.036059 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9e325-cde4-4533-8a32-5694ff66d94d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:21.036334 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd9e325-cde4-4533-8a32-5694ff66d94d None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:21.037770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd9e325-cde4-4533-8a32-5694ff66d94d None admin] RBAC: Adding query filter params (name=member) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:21.038093 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd9e325-cde4-4533-8a32-5694ff66d94d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:21.039782 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd9e325-cde4-4533-8a32-5694ff66d94d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:21.058414 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 151/290] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jul 13 23:01:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:01:21.064949 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ca098-d384-4c26-bc77-2c3cd7f3b1f4 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:21.118791 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:21.227416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ca098-d384-4c26-bc77-2c3cd7f3b1f4 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:21.273264 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b27e48-6b6a-469d-95a7-8201a6985788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:21.274199 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b27e48-6b6a-469d-95a7-8201a6985788 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:21.274488 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b27e48-6b6a-469d-95a7-8201a6985788 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:21.274764 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b27e48-6b6a-469d-95a7-8201a6985788 None admin] PATH_INFO: `/v3/users/ea11955c44dd41cc998c143ef68f43d2` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:21.276160 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b27e48-6b6a-469d-95a7-8201a6985788 None admin] RBAC: Authorizing `identity:get_user(user_id=ea11955c44dd41cc998c143ef68f43d2)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:21.277689 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b27e48-6b6a-469d-95a7-8201a6985788 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:21.279402 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 140/291] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:01:21 2022] GET /identity/v3/users/ea11955c44dd41cc998c143ef68f43d2 => generated 297 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:21.285083 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd9e325-cde4-4533-8a32-5694ff66d94d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:21.412365 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd9e325-cde4-4533-8a32-5694ff66d94d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:21.454714 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b44d8f-17ad-4d5c-a160-6be76b541028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:21.455391 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b44d8f-17ad-4d5c-a160-6be76b541028 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:21.455704 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b44d8f-17ad-4d5c-a160-6be76b541028 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:21.455987 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b44d8f-17ad-4d5c-a160-6be76b541028 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:21.485645 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:21.534772 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b44d8f-17ad-4d5c-a160-6be76b541028 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:21.536347 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b44d8f-17ad-4d5c-a160-6be76b541028 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:21.648034 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 152/292] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:01:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 23:01:21.654556 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b27e48-6b6a-469d-95a7-8201a6985788 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:21.823845 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b27e48-6b6a-469d-95a7-8201a6985788 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:21.826732 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:21.872246 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b2e30-8e2e-462a-a7ee-cb2547c07b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:21.873651 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b2e30-8e2e-462a-a7ee-cb2547c07b44 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:21.874186 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b2e30-8e2e-462a-a7ee-cb2547c07b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:21.874702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b2e30-8e2e-462a-a7ee-cb2547c07b44 None admin] PATH_INFO: `/v3/projects/b64d10f5c7424d909a79d57916eaeac5` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:21.876250 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78b2e30-8e2e-462a-a7ee-cb2547c07b44 None admin] RBAC: Authorizing `identity:get_project(project_id=b64d10f5c7424d909a79d57916eaeac5)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:21.877918 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78b2e30-8e2e-462a-a7ee-cb2547c07b44 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:21.879883 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 141/293] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:01:21 2022] GET /identity/v3/projects/b64d10f5c7424d909a79d57916eaeac5 => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:21.886345 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b44d8f-17ad-4d5c-a160-6be76b541028 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:21.980430 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:22.031861 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b44d8f-17ad-4d5c-a160-6be76b541028 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:22.075445 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b9e6b-0282-40a7-9a74-07855147831f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:22.076078 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b9e6b-0282-40a7-9a74-07855147831f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:22.076357 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b9e6b-0282-40a7-9a74-07855147831f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:22.076655 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b9e6b-0282-40a7-9a74-07855147831f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:22.077524 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5b9e6b-0282-40a7-9a74-07855147831f None admin] RBAC: Adding query filter params (role.id=f005bdf39a0743238b3476f6b88b3684, scope.project.id=b64d10f5c7424d909a79d57916eaeac5, user.id=ea11955c44dd41cc998c143ef68f43d2) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:22.077832 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5b9e6b-0282-40a7-9a74-07855147831f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:22.079340 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5b9e6b-0282-40a7-9a74-07855147831f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:22.090969 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 153/294] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:01:21 2022] GET /identity/v3/role_assignments?user.id=ea11955c44dd41cc998c143ef68f43d2&scope.project.id=b64d10f5c7424d909a79d57916eaeac5&role.id=f005bdf39a0743238b3476f6b88b3684 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:22.801859 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e038e9e-83cf-4b42-a24e-a2337d2ea7fc None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:22.802554 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e038e9e-83cf-4b42-a24e-a2337d2ea7fc None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:22.802554 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e038e9e-83cf-4b42-a24e-a2337d2ea7fc None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:22.804456 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 142/295] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:22.810648 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c3b73-7c5e-41c6-935a-5b382fe51840 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:22.810931 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c3b73-7c5e-41c6-935a-5b382fe51840 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:22.811204 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c3b73-7c5e-41c6-935a-5b382fe51840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:22.830357 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-8e0c3b73-7c5e-41c6-935a-5b382fe51840 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:22.831591 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8e0c3b73-7c5e-41c6-935a-5b382fe51840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:22.916106 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 154/296] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:22.938719 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e038e9e-83cf-4b42-a24e-a2337d2ea7fc None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:22.951939 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01613a-01b0-42c5-ae93-de7e6ac98d9d None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:22.952238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01613a-01b0-42c5-ae93-de7e6ac98d9d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:22.952490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01613a-01b0-42c5-ae93-de7e6ac98d9d None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:22.954565 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 155/297] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:01:22.961300 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b5d1d-43c7-4ad9-8ddc-bfb178636110 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:22.961780 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b5d1d-43c7-4ad9-8ddc-bfb178636110 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:22.962064 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b5d1d-43c7-4ad9-8ddc-bfb178636110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:22.983105 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-652b5d1d-43c7-4ad9-8ddc-bfb178636110 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:22.984885 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-652b5d1d-43c7-4ad9-8ddc-bfb178636110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:23.041801 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:23.065957 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e038e9e-83cf-4b42-a24e-a2337d2ea7fc None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:23.066244 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 156/298] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:23.073886 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546cb29-9aae-4e68-9e9a-ca6e8568814d None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:23.074956 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546cb29-9aae-4e68-9e9a-ca6e8568814d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:23.075439 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546cb29-9aae-4e68-9e9a-ca6e8568814d None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:23.077505 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 157/299] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:23 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:23.084171 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76717f6-12b3-442e-82c4-7f10667c72f9 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:23.085116 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76717f6-12b3-442e-82c4-7f10667c72f9 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:23.086078 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76717f6-12b3-442e-82c4-7f10667c72f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:23.104621 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-e76717f6-12b3-442e-82c4-7f10667c72f9 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:23.104937 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb994e07-1f8a-41c1-9435-3c65e11cef0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:23.105648 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb994e07-1f8a-41c1-9435-3c65e11cef0b None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:23.105823 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e76717f6-12b3-442e-82c4-7f10667c72f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:23.105942 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb994e07-1f8a-41c1-9435-3c65e11cef0b None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:23.106181 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb994e07-1f8a-41c1-9435-3c65e11cef0b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:23.107351 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb994e07-1f8a-41c1-9435-3c65e11cef0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:23.109955 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb994e07-1f8a-41c1-9435-3c65e11cef0b None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:23.116150 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 143/300] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:01:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:23.116150 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb994e07-1f8a-41c1-9435-3c65e11cef0b None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:23.163854 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:23.182027 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 158/301] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:23.182755 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76717f6-12b3-442e-82c4-7f10667c72f9 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:23.234021 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb994e07-1f8a-41c1-9435-3c65e11cef0b None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:23.276869 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57689c17-6f8a-4eb3-975d-35f4a3aae7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:23.277669 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57689c17-6f8a-4eb3-975d-35f4a3aae7ca None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:23.277949 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57689c17-6f8a-4eb3-975d-35f4a3aae7ca None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:23.278249 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57689c17-6f8a-4eb3-975d-35f4a3aae7ca None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:23.290196 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57689c17-6f8a-4eb3-975d-35f4a3aae7ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:23.298564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-57689c17-6f8a-4eb3-975d-35f4a3aae7ca None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 23:01:23.298564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:23.298564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:23.298564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:23.298564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:23.298564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:23.298564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:23.298564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:23.298564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:23.298564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:23.298564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:23.298564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:23.298564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:23.298564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:23.298564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:23.298564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:23.299457 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:23.299457 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:23.299457 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:23.299457 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:23.299457 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:23.299457 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:23.299457 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:23.299457 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:23.299457 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:23.299457 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:23.299457 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:23.299457 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:23.299457 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:23.299457 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:23.299457 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:23.299457 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:23.300719 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:23.300719 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:23.300719 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:23.300719 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:23.300719 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:23.300719 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:23.300719 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:23.300719 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:23.300719 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:23.300719 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:23.300719 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 23:01:23.300719 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:23.300719 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 144/302] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:01:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:23.300719 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57689c17-6f8a-4eb3-975d-35f4a3aae7ca None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:23.305054 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76717f6-12b3-442e-82c4-7f10667c72f9 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:23.317413 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:23.343192 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf8298f-cf1d-491b-b612-bbf06ece4ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:23.343900 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8298f-cf1d-491b-b612-bbf06ece4ca7 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:23.344483 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8298f-cf1d-491b-b612-bbf06ece4ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:23.344483 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8298f-cf1d-491b-b612-bbf06ece4ca7 None admin] PATH_INFO: `/v3/groups` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:23.345111 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf8298f-cf1d-491b-b612-bbf06ece4ca7 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:23.346681 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf8298f-cf1d-491b-b612-bbf06ece4ca7 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:23.368109 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 159/303] 10.222.0.60 () {60 vars in 1134 bytes} [Wed Jul 13 23:01:23 2022] POST /identity/v3/groups => generated 229 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:01:23.376607 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf8298f-cf1d-491b-b612-bbf06ece4ca7 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:23.417083 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:23.420244 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57689c17-6f8a-4eb3-975d-35f4a3aae7ca None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:23.462837 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc41962-3e99-43f5-9a77-facb42704b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:23.463740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc41962-3e99-43f5-9a77-facb42704b35 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:23.464146 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc41962-3e99-43f5-9a77-facb42704b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:23.464711 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc41962-3e99-43f5-9a77-facb42704b35 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:23.477175 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc41962-3e99-43f5-9a77-facb42704b35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:23.486225 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-5bc41962-3e99-43f5-9a77-facb42704b35 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 23:01:23.486225 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:23.486225 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:23.486225 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:23.486225 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:23.486225 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:23.486225 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:23.486225 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:23.486225 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:23.486225 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:23.486225 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:23.486225 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:23.486225 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:23.486225 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:23.486225 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:23.486225 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:23.487806 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:23.487806 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:23.487806 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:23.487806 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:23.487806 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:23.487806 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:23.487806 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:23.487806 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:23.487806 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:23.487806 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:23.487806 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:23.487806 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:23.487806 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:23.487806 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:23.487806 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:23.487806 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:23.489156 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:23.489156 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:23.489156 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:23.489156 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:23.489156 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:23.489156 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:23.489156 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:23.489156 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:23.489156 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:23.489156 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:23.489156 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 23:01:23.489156 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:23.490499 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 145/304] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:01:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:23.497467 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc41962-3e99-43f5-9a77-facb42704b35 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:23.544902 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf8298f-cf1d-491b-b612-bbf06ece4ca7 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:23.582596 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de899f4e-263b-401a-9bbf-1d36300af78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:23.583308 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de899f4e-263b-401a-9bbf-1d36300af78a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:23.583731 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de899f4e-263b-401a-9bbf-1d36300af78a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:23.584081 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de899f4e-263b-401a-9bbf-1d36300af78a None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:23.584849 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de899f4e-263b-401a-9bbf-1d36300af78a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:23.585158 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de899f4e-263b-401a-9bbf-1d36300af78a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:23.586568 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de899f4e-263b-401a-9bbf-1d36300af78a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:23.596223 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 160/305] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jul 13 23:01:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:23.597428 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef62a9-a207-4eb7-a7f8-98e0a4acdc45 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:23.597723 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef62a9-a207-4eb7-a7f8-98e0a4acdc45 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:23.598026 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef62a9-a207-4eb7-a7f8-98e0a4acdc45 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:23.599441 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 161/306] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:23.604539 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef62a9-a207-4eb7-a7f8-98e0a4acdc45 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:23.632645 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc41962-3e99-43f5-9a77-facb42704b35 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:23.639273 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:23.671684 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaa39c0-6787-4db3-9a8a-0ecc0e147437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:23.672359 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa39c0-6787-4db3-9a8a-0ecc0e147437 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:23.672622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa39c0-6787-4db3-9a8a-0ecc0e147437 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:23.672895 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa39c0-6787-4db3-9a8a-0ecc0e147437 None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:23.673642 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaa39c0-6787-4db3-9a8a-0ecc0e147437 None admin] RBAC: Adding query filter params (name=member) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:23.673945 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaa39c0-6787-4db3-9a8a-0ecc0e147437 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:23.675663 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaa39c0-6787-4db3-9a8a-0ecc0e147437 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:23.684592 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 146/307] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jul 13 23:01:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:23.685885 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9650aa-1346-41de-980b-6825c7242b83 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:23.686172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9650aa-1346-41de-980b-6825c7242b83 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:23.686443 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9650aa-1346-41de-980b-6825c7242b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:23.703295 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-0a9650aa-1346-41de-980b-6825c7242b83 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:23.704540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-0a9650aa-1346-41de-980b-6825c7242b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:23.723281 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:23.723966 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef62a9-a207-4eb7-a7f8-98e0a4acdc45 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:23.760598 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c4c424-7f06-4c69-9467-0bacd6f55b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:23.761581 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4c424-7f06-4c69-9467-0bacd6f55b36 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:23.761925 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4c424-7f06-4c69-9467-0bacd6f55b36 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:23.762214 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4c424-7f06-4c69-9467-0bacd6f55b36 None admin] PATH_INFO: `/v3/users/0a98f73270bd4ce699b3f0bbd6c9f9c0` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:23.778186 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c4c424-7f06-4c69-9467-0bacd6f55b36 None admin] RBAC: Authorizing `identity:get_user(user_id=0a98f73270bd4ce699b3f0bbd6c9f9c0)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:23.779901 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c4c424-7f06-4c69-9467-0bacd6f55b36 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:23.786598 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 162/308] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:01:23 2022] GET /identity/v3/users/0a98f73270bd4ce699b3f0bbd6c9f9c0 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:23.786598 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c4c424-7f06-4c69-9467-0bacd6f55b36 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:23.788262 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 147/309] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:23.789023 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9650aa-1346-41de-980b-6825c7242b83 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:23.891827 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:23.903191 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c4c424-7f06-4c69-9467-0bacd6f55b36 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:23.909575 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9650aa-1346-41de-980b-6825c7242b83 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:23.946500 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a701e13-5a56-47f3-a775-f37409740968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:23.948191 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf5995a-5269-41dd-a03c-d56f1114668b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:23.948303 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a701e13-5a56-47f3-a775-f37409740968 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:23.948303 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a701e13-5a56-47f3-a775-f37409740968 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:23.948457 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5995a-5269-41dd-a03c-d56f1114668b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:23.948522 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a701e13-5a56-47f3-a775-f37409740968 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:23.948586 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5995a-5269-41dd-a03c-d56f1114668b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:23.948586 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5995a-5269-41dd-a03c-d56f1114668b None admin] PATH_INFO: `/v3/users/5e56551e804645dd906ee799dd69b727` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:23.948861 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a701e13-5a56-47f3-a775-f37409740968 None admin] RBAC: Adding query filter params (role.id=2562dfc9fd0845678cb5beff6a715255, user.id=0a98f73270bd4ce699b3f0bbd6c9f9c0, scope.system=all) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:23.949122 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a701e13-5a56-47f3-a775-f37409740968 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:23.949722 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf5995a-5269-41dd-a03c-d56f1114668b None admin] RBAC: Authorizing `identity:get_user(user_id=5e56551e804645dd906ee799dd69b727)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:23.950567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a701e13-5a56-47f3-a775-f37409740968 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:23.951174 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf5995a-5269-41dd-a03c-d56f1114668b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:23.953042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 163/310] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:01:23 2022] GET /identity/v3/users/5e56551e804645dd906ee799dd69b727 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:23.953663 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf5995a-5269-41dd-a03c-d56f1114668b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:23.964865 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 148/311] 10.222.0.60 () {56 vars in 1308 bytes} [Wed Jul 13 23:01:23 2022] GET /identity/v3/role_assignments?user.id=0a98f73270bd4ce699b3f0bbd6c9f9c0&scope.system=all&role.id=2562dfc9fd0845678cb5beff6a715255 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:23.964865 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a701e13-5a56-47f3-a775-f37409740968 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:23.988756 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:24.108440 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a701e13-5a56-47f3-a775-f37409740968 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:24.109322 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf5995a-5269-41dd-a03c-d56f1114668b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:24.179816 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8831f-0d3e-4598-bb58-7b49f7de4752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:24.180674 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8831f-0d3e-4598-bb58-7b49f7de4752 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:24.180954 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8831f-0d3e-4598-bb58-7b49f7de4752 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:24.181424 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8831f-0d3e-4598-bb58-7b49f7de4752 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:24.182696 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc381e-b625-45d8-a8ef-bda12b4c7fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:24.184795 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc381e-b625-45d8-a8ef-bda12b4c7fb1 None admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:24.185511 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc381e-b625-45d8-a8ef-bda12b4c7fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:24.186054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdc381e-b625-45d8-a8ef-bda12b4c7fb1 None admin] PATH_INFO: `/v3/system/users/5e56551e804645dd906ee799dd69b727/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:24.188124 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdc381e-b625-45d8-a8ef-bda12b4c7fb1 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5e56551e804645dd906ee799dd69b727, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:24.192540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdc381e-b625-45d8-a8ef-bda12b4c7fb1 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:24.194713 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a8831f-0d3e-4598-bb58-7b49f7de4752 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:24.205208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-10a8831f-0d3e-4598-bb58-7b49f7de4752 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 23:01:24.205208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:24.205208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:24.205208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:24.205208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:24.205208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:24.205208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:24.205208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:24.205208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:24.205208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:24.205208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:24.205208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:24.205208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:24.205208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:24.205208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:24.205208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:24.206266 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:24.206266 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:24.206266 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:24.206266 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:24.206266 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:24.206266 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:24.206266 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:24.206266 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:24.206266 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:24.206266 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:24.206266 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:24.206266 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:24.206266 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:24.206266 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:24.206266 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:24.206266 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:24.207042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:24.207042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:24.207042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:24.207042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:24.207042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:24.207042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:24.207042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:24.207042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:24.207042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:24.207042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:24.207042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 23:01:24.207042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:24.207042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 164/312] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:01:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:24.209978 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 149/313] 10.222.0.60 () {58 vars in 1334 bytes} [Wed Jul 13 23:01:23 2022] PUT /identity/v3/system/users/5e56551e804645dd906ee799dd69b727/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:01:24.220709 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8831f-0d3e-4598-bb58-7b49f7de4752 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:24.251762 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:24.385117 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8831f-0d3e-4598-bb58-7b49f7de4752 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:24.457942 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33ed561-3995-4931-9def-676655630561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:24.458959 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ed561-3995-4931-9def-676655630561 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:24.459421 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ed561-3995-4931-9def-676655630561 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:24.459935 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ed561-3995-4931-9def-676655630561 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:24.461339 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33ed561-3995-4931-9def-676655630561 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:24.461890 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33ed561-3995-4931-9def-676655630561 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:24.464618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33ed561-3995-4931-9def-676655630561 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:24.481111 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 165/314] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jul 13 23:01:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:24.486727 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc381e-b625-45d8-a8ef-bda12b4c7fb1 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:24.587754 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:24.614169 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdc381e-b625-45d8-a8ef-bda12b4c7fb1 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:24.654033 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eab9b0-ebba-4bde-8a55-5750e9158b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:24.655277 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eab9b0-ebba-4bde-8a55-5750e9158b29 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:24.655823 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eab9b0-ebba-4bde-8a55-5750e9158b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:24.656301 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eab9b0-ebba-4bde-8a55-5750e9158b29 None admin] PATH_INFO: `/v3/users/7eb8a38ea83b488fae09e62392d9b449` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:24.657818 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eab9b0-ebba-4bde-8a55-5750e9158b29 None admin] RBAC: Authorizing `identity:get_user(user_id=7eb8a38ea83b488fae09e62392d9b449)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:24.659525 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eab9b0-ebba-4bde-8a55-5750e9158b29 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:24.661977 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 150/315] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:01:24 2022] GET /identity/v3/users/7eb8a38ea83b488fae09e62392d9b449 => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:24.667882 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33ed561-3995-4931-9def-676655630561 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:24.766375 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:24.793164 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33ed561-3995-4931-9def-676655630561 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:24.845719 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136b2726-335c-4cec-8c27-7b1bdbb36a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:24.846527 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b2726-335c-4cec-8c27-7b1bdbb36a74 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:24.846847 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b2726-335c-4cec-8c27-7b1bdbb36a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:24.847155 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b2726-335c-4cec-8c27-7b1bdbb36a74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:24.927284 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136b2726-335c-4cec-8c27-7b1bdbb36a74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:24.929145 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136b2726-335c-4cec-8c27-7b1bdbb36a74 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:25.016293 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:25.040329 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 166/316] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:01:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 23:01:25.047017 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eab9b0-ebba-4bde-8a55-5750e9158b29 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:25.163350 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eab9b0-ebba-4bde-8a55-5750e9158b29 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:25.202029 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d76227-d429-4588-a729-c668d04facb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:25.203275 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d76227-d429-4588-a729-c668d04facb9 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:25.203789 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d76227-d429-4588-a729-c668d04facb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:25.204268 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d76227-d429-4588-a729-c668d04facb9 None admin] PATH_INFO: `/v3/projects/b64d10f5c7424d909a79d57916eaeac5` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:25.205826 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d76227-d429-4588-a729-c668d04facb9 None admin] RBAC: Authorizing `identity:get_project(project_id=b64d10f5c7424d909a79d57916eaeac5)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:25.207812 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d76227-d429-4588-a729-c668d04facb9 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:25.210148 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 151/317] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:01:25 2022] GET /identity/v3/projects/b64d10f5c7424d909a79d57916eaeac5 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:25.216803 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136b2726-335c-4cec-8c27-7b1bdbb36a74 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:25.332236 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136b2726-335c-4cec-8c27-7b1bdbb36a74 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:25.372018 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3061fe-ce69-48bc-a82a-cef67daeb39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:25.372692 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3061fe-ce69-48bc-a82a-cef67daeb39a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:25.372973 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3061fe-ce69-48bc-a82a-cef67daeb39a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:25.373249 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3061fe-ce69-48bc-a82a-cef67daeb39a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:25.374096 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3061fe-ce69-48bc-a82a-cef67daeb39a None admin] RBAC: Adding query filter params (role.id=2562dfc9fd0845678cb5beff6a715255, scope.project.id=b64d10f5c7424d909a79d57916eaeac5, user.id=7eb8a38ea83b488fae09e62392d9b449) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:25.374406 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3061fe-ce69-48bc-a82a-cef67daeb39a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:25.375908 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3061fe-ce69-48bc-a82a-cef67daeb39a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:25.383111 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 167/318] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:01:25 2022] GET /identity/v3/role_assignments?user.id=7eb8a38ea83b488fae09e62392d9b449&scope.project.id=b64d10f5c7424d909a79d57916eaeac5&role.id=2562dfc9fd0845678cb5beff6a715255 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:25.525670 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54501fb-6468-45e6-89c1-66f7930c9983 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:25.526041 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54501fb-6468-45e6-89c1-66f7930c9983 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:25.526240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54501fb-6468-45e6-89c1-66f7930c9983 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:25.528262 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 152/319] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:25.534392 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3681155-8029-416d-900a-882993382f15 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:25.534711 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3681155-8029-416d-900a-882993382f15 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:25.534945 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3681155-8029-416d-900a-882993382f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:25.552991 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-f3681155-8029-416d-900a-882993382f15 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:25.554237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f3681155-8029-416d-900a-882993382f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:25.601494 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:25.631234 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 168/320] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:25.656072 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54501fb-6468-45e6-89c1-66f7930c9983 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:25.676542 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:25.774184 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54501fb-6468-45e6-89c1-66f7930c9983 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:25.822491 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54011aa-8bc9-4ddf-9395-450e2b3279d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:25.823563 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54011aa-8bc9-4ddf-9395-450e2b3279d6 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:25.823863 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54011aa-8bc9-4ddf-9395-450e2b3279d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:25.824133 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54011aa-8bc9-4ddf-9395-450e2b3279d6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:25.825529 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54011aa-8bc9-4ddf-9395-450e2b3279d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:25.827036 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54011aa-8bc9-4ddf-9395-450e2b3279d6 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:25.829078 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 153/321] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:01:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:25.834884 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3681155-8029-416d-900a-882993382f15 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:25.974282 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3681155-8029-416d-900a-882993382f15 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:26.018659 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734c74e-8c13-4341-aec4-c1f9fbf41324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:26.019767 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734c74e-8c13-4341-aec4-c1f9fbf41324 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:26.020083 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734c74e-8c13-4341-aec4-c1f9fbf41324 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:26.020361 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734c74e-8c13-4341-aec4-c1f9fbf41324 None admin] PATH_INFO: `/v3/groups` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:26.021397 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e734c74e-8c13-4341-aec4-c1f9fbf41324 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:26.023373 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e734c74e-8c13-4341-aec4-c1f9fbf41324 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:26.045010 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 169/322] 10.222.0.60 () {60 vars in 1134 bytes} [Wed Jul 13 23:01:25 2022] POST /identity/v3/groups => generated 226 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:01:26.200686 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756735b-faf8-45c6-a4ce-d7d6ef3ba424 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:26.201380 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756735b-faf8-45c6-a4ce-d7d6ef3ba424 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:26.201380 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756735b-faf8-45c6-a4ce-d7d6ef3ba424 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:26.202904 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 154/323] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:01:26.208830 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501dbe3b-a422-45fd-b60d-6ffc0f9a1685 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:26.209100 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501dbe3b-a422-45fd-b60d-6ffc0f9a1685 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:26.209399 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501dbe3b-a422-45fd-b60d-6ffc0f9a1685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:26.229626 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-501dbe3b-a422-45fd-b60d-6ffc0f9a1685 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:26.230766 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-501dbe3b-a422-45fd-b60d-6ffc0f9a1685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:26.264634 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:26.314951 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 170/324] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:26.338547 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3756735b-faf8-45c6-a4ce-d7d6ef3ba424 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:26.431123 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:26.453728 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3756735b-faf8-45c6-a4ce-d7d6ef3ba424 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:26.491997 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f0cb9c-0bdf-427c-8ead-0c0f6a0bf680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:26.492762 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f0cb9c-0bdf-427c-8ead-0c0f6a0bf680 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:26.493041 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f0cb9c-0bdf-427c-8ead-0c0f6a0bf680 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:26.493316 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f0cb9c-0bdf-427c-8ead-0c0f6a0bf680 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:26.505540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f0cb9c-0bdf-427c-8ead-0c0f6a0bf680 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:26.515858 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb28d6-a143-4887-8c39-de9313144a8b None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:26.516288 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb28d6-a143-4887-8c39-de9313144a8b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:26.516387 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-26f0cb9c-0bdf-427c-8ead-0c0f6a0bf680 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 23:01:26.516387 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:26.516387 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:26.516387 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:26.516387 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:26.516387 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:26.516387 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:26.516387 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:26.516387 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:26.516387 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:26.516387 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:26.516387 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:26.516387 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:26.516387 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:26.516387 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:26.516387 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:26.517201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:26.517201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:26.517201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:26.517201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:26.517201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:26.517201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:26.517201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:26.517201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:26.517201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:26.517201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:26.517201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:26.517201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:26.517201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:26.517201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:26.517201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:26.517201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:26.517968 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:26.517968 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:26.517968 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:26.517968 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:26.517968 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:26.517968 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:26.517968 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:26.517968 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:26.517968 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:26.517968 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:26.517968 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 23:01:26.517968 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:26.518562 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb28d6-a143-4887-8c39-de9313144a8b None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:26.518630 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 155/325] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:01:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:26.520182 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 171/326] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:26.526599 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fb28d6-a143-4887-8c39-de9313144a8b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:26.527007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e04e7-c76b-45e6-b2ad-a479c7045fd2 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:26.527301 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e04e7-c76b-45e6-b2ad-a479c7045fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:26.527603 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e04e7-c76b-45e6-b2ad-a479c7045fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:26.548742 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-8f0e04e7-c76b-45e6-b2ad-a479c7045fd2 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:26.549968 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-8f0e04e7-c76b-45e6-b2ad-a479c7045fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:26.631814 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 156/327] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:26.659387 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e04e7-c76b-45e6-b2ad-a479c7045fd2 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:26.690890 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fb28d6-a143-4887-8c39-de9313144a8b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:26.736584 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b27d4-08bf-4c84-8e6e-dc82fa5ec2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:26.737448 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b27d4-08bf-4c84-8e6e-dc82fa5ec2ab None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:26.737837 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b27d4-08bf-4c84-8e6e-dc82fa5ec2ab None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:26.738214 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b27d4-08bf-4c84-8e6e-dc82fa5ec2ab None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:26.739108 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37b27d4-08bf-4c84-8e6e-dc82fa5ec2ab None admin] RBAC: Adding query filter params (name=reader) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:26.739516 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37b27d4-08bf-4c84-8e6e-dc82fa5ec2ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:26.741547 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37b27d4-08bf-4c84-8e6e-dc82fa5ec2ab None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:26.751951 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 172/328] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jul 13 23:01:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:01:26.759200 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b27d4-08bf-4c84-8e6e-dc82fa5ec2ab None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:26.760501 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:26.793517 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:26.794515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e04e7-c76b-45e6-b2ad-a479c7045fd2 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:26.843866 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a15c04-2e8f-46c6-ba05-7af5b699a89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:26.845733 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a15c04-2e8f-46c6-ba05-7af5b699a89f None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:26.846160 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a15c04-2e8f-46c6-ba05-7af5b699a89f None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:26.846865 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a15c04-2e8f-46c6-ba05-7af5b699a89f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:26.870851 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a15c04-2e8f-46c6-ba05-7af5b699a89f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:26.892771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-85a15c04-2e8f-46c6-ba05-7af5b699a89f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 23:01:26.892771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:26.892771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:26.892771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:26.892771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:26.892771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:26.892771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:26.892771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:26.892771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:26.892771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:26.892771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:26.892771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:26.892771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:26.892771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:26.892771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:26.892771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:26.894669 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:26.894669 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:26.894669 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:26.894669 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:26.894669 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:26.894669 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:26.894669 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:26.894669 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:26.894669 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:26.894669 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:26.894669 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:26.894669 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:26.894669 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:26.894669 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:26.894669 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:26.894669 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:26.896345 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:26.896345 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:26.896345 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:26.896345 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:26.896345 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:26.896345 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:26.896345 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:26.896345 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:26.896345 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:26.896345 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:26.896345 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 23:01:26.896345 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:26.897484 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 157/329] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:01:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:01:26.908245 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a15c04-2e8f-46c6-ba05-7af5b699a89f None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:26.933913 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b27d4-08bf-4c84-8e6e-dc82fa5ec2ab None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:26.974011 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93dd52-0bf6-4c35-b8b7-4cf7754c4357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:26.975259 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93dd52-0bf6-4c35-b8b7-4cf7754c4357 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:26.975807 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93dd52-0bf6-4c35-b8b7-4cf7754c4357 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:26.976305 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93dd52-0bf6-4c35-b8b7-4cf7754c4357 None admin] PATH_INFO: `/v3/users/0a98f73270bd4ce699b3f0bbd6c9f9c0` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:26.978332 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b93dd52-0bf6-4c35-b8b7-4cf7754c4357 None admin] RBAC: Authorizing `identity:get_user(user_id=0a98f73270bd4ce699b3f0bbd6c9f9c0)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:26.980186 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b93dd52-0bf6-4c35-b8b7-4cf7754c4357 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:26.982568 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 173/330] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:01:26 2022] GET /identity/v3/users/0a98f73270bd4ce699b3f0bbd6c9f9c0 => generated 293 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:26.989416 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93dd52-0bf6-4c35-b8b7-4cf7754c4357 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:27.096164 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:27.106068 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93dd52-0bf6-4c35-b8b7-4cf7754c4357 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:27.106765 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a15c04-2e8f-46c6-ba05-7af5b699a89f None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:27.173280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55465be1-8808-4101-ad1d-85359ba73406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:27.174410 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55465be1-8808-4101-ad1d-85359ba73406 None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:27.174710 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55465be1-8808-4101-ad1d-85359ba73406 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:27.174813 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f733124a-7d37-41a1-ae3a-b5f85c028581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:27.175233 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55465be1-8808-4101-ad1d-85359ba73406 None admin] PATH_INFO: `/v3/system/users/0a98f73270bd4ce699b3f0bbd6c9f9c0/roles/2562dfc9fd0845678cb5beff6a715255` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:27.175440 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733124a-7d37-41a1-ae3a-b5f85c028581 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:27.175749 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733124a-7d37-41a1-ae3a-b5f85c028581 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:27.176027 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733124a-7d37-41a1-ae3a-b5f85c028581 None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:27.176795 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f733124a-7d37-41a1-ae3a-b5f85c028581 None admin] RBAC: Adding query filter params (name=member) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:27.177102 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f733124a-7d37-41a1-ae3a-b5f85c028581 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:27.178530 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f733124a-7d37-41a1-ae3a-b5f85c028581 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:27.180206 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:27.182760 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55465be1-8808-4101-ad1d-85359ba73406 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0a98f73270bd4ce699b3f0bbd6c9f9c0, role_id=2562dfc9fd0845678cb5beff6a715255)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:27.185108 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55465be1-8808-4101-ad1d-85359ba73406 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:27.200448 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 158/331] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jul 13 23:01:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:27.202715 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 174/332] 10.222.0.60 () {58 vars in 1334 bytes} [Wed Jul 13 23:01:26 2022] PUT /identity/v3/system/users/0a98f73270bd4ce699b3f0bbd6c9f9c0/roles/2562dfc9fd0845678cb5beff6a715255 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:01:27.212366 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f733124a-7d37-41a1-ae3a-b5f85c028581 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:27.411028 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f733124a-7d37-41a1-ae3a-b5f85c028581 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:27.453659 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952a5353-44ff-448f-94d9-4f45f91a32f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:27.454558 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a5353-44ff-448f-94d9-4f45f91a32f1 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:27.454844 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a5353-44ff-448f-94d9-4f45f91a32f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:27.455124 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a5353-44ff-448f-94d9-4f45f91a32f1 None admin] PATH_INFO: `/v3/users/5e56551e804645dd906ee799dd69b727` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:27.458954 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952a5353-44ff-448f-94d9-4f45f91a32f1 None admin] RBAC: Authorizing `identity:get_user(user_id=5e56551e804645dd906ee799dd69b727)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:27.460493 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952a5353-44ff-448f-94d9-4f45f91a32f1 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:27.463125 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 159/333] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:01:27 2022] GET /identity/v3/users/5e56551e804645dd906ee799dd69b727 => generated 293 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:27.469097 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55465be1-8808-4101-ad1d-85359ba73406 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:27.598622 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55465be1-8808-4101-ad1d-85359ba73406 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:27.641434 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30999cee-bae8-4691-b5aa-54f8c92f74bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:27.642116 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30999cee-bae8-4691-b5aa-54f8c92f74bd None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:27.642400 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30999cee-bae8-4691-b5aa-54f8c92f74bd None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:27.642677 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30999cee-bae8-4691-b5aa-54f8c92f74bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:27.643525 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30999cee-bae8-4691-b5aa-54f8c92f74bd None admin] RBAC: Adding query filter params (role.id=f005bdf39a0743238b3476f6b88b3684, user.id=5e56551e804645dd906ee799dd69b727, scope.system=all) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:27.643861 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30999cee-bae8-4691-b5aa-54f8c92f74bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:27.645559 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30999cee-bae8-4691-b5aa-54f8c92f74bd None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:27.663187 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 175/334] 10.222.0.60 () {56 vars in 1308 bytes} [Wed Jul 13 23:01:27 2022] GET /identity/v3/role_assignments?user.id=5e56551e804645dd906ee799dd69b727&scope.system=all&role.id=f005bdf39a0743238b3476f6b88b3684 => generated 510 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:01:29.174756 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f2d69-829b-4d63-8ac9-aa04b1ae4663 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:29.175652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f2d69-829b-4d63-8ac9-aa04b1ae4663 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:29.175652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f2d69-829b-4d63-8ac9-aa04b1ae4663 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:29.177081 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 160/335] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:29.196640 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ab0a7-294d-4724-9c7b-3349686d0780 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:29.197472 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ab0a7-294d-4724-9c7b-3349686d0780 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:29.197472 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ab0a7-294d-4724-9c7b-3349686d0780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:29.222571 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-e68ab0a7-294d-4724-9c7b-3349686d0780 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:29.223836 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e68ab0a7-294d-4724-9c7b-3349686d0780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:29.238523 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:29.309021 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 176/336] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:29.387610 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52f2d69-829b-4d63-8ac9-aa04b1ae4663 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:29.468151 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:29.510921 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52f2d69-829b-4d63-8ac9-aa04b1ae4663 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:29.596690 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a7d68-771b-469f-9bc8-baf94ff53027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:29.598575 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a7d68-771b-469f-9bc8-baf94ff53027 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:29.599367 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a7d68-771b-469f-9bc8-baf94ff53027 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:29.600173 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a7d68-771b-469f-9bc8-baf94ff53027 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:29.627336 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a7d68-771b-469f-9bc8-baf94ff53027 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:29.645870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-3e6a7d68-771b-469f-9bc8-baf94ff53027 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 23:01:29.645870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:29.645870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:29.645870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:29.645870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:29.645870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:29.645870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:29.645870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:29.645870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:29.645870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:29.645870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:29.645870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:29.645870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:29.645870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:29.645870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:29.645870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:29.649202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:29.649202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:29.649202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:29.649202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:29.649202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:29.649202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:29.649202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:29.649202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:29.649202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:29.649202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:29.649202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:29.649202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:29.649202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:29.649202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:29.649202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:29.649202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:29.651122 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:29.651122 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:29.651122 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:29.651122 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:29.651122 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:29.651122 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:29.651122 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:29.651122 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:29.651122 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:29.651122 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:29.651122 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 23:01:29.651122 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:29.655494 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 161/337] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:01:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:01:29.662275 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f664c-f2d6-47b5-8c3e-25ab78ee00cc None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:29.662567 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f664c-f2d6-47b5-8c3e-25ab78ee00cc None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:29.662843 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f664c-f2d6-47b5-8c3e-25ab78ee00cc None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:29.664535 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 177/338] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:29.666476 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a7d68-771b-469f-9bc8-baf94ff53027 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:29.669685 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087dd159-0f1c-458f-b026-48223f6e2b3f None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:29.669972 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087dd159-0f1c-458f-b026-48223f6e2b3f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:29.670255 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087dd159-0f1c-458f-b026-48223f6e2b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:29.720013 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-087dd159-0f1c-458f-b026-48223f6e2b3f None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:29.726266 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-087dd159-0f1c-458f-b026-48223f6e2b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:29.919886 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 178/339] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:29.924037 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c47cd-1d12-4f4f-b94e-b6ff14e937ac None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:29.924758 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c47cd-1d12-4f4f-b94e-b6ff14e937ac None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:29.925470 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c47cd-1d12-4f4f-b94e-b6ff14e937ac None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:29.928505 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 179/340] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:29.936064 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f59268-2945-4b01-9e64-e4335447c5cc None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:29.936803 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f59268-2945-4b01-9e64-e4335447c5cc None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:29.937519 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f59268-2945-4b01-9e64-e4335447c5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:29.953235 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a7d68-771b-469f-9bc8-baf94ff53027 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:29.994088 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-31f59268-2945-4b01-9e64-e4335447c5cc None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:29.998672 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-31f59268-2945-4b01-9e64-e4335447c5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:30.041119 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:30.062881 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc74c7-1d60-4116-960a-86fbf40064d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:30.063573 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc74c7-1d60-4116-960a-86fbf40064d4 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:30.063869 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc74c7-1d60-4116-960a-86fbf40064d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:30.064140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc74c7-1d60-4116-960a-86fbf40064d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:30.064977 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fc74c7-1d60-4116-960a-86fbf40064d4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:30.065270 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fc74c7-1d60-4116-960a-86fbf40064d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:30.066663 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fc74c7-1d60-4116-960a-86fbf40064d4 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:30.079473 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 162/341] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jul 13 23:01:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:30.080328 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc74c7-1d60-4116-960a-86fbf40064d4 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:30.175491 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 180/342] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 23:01:30.177532 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f59268-2945-4b01-9e64-e4335447c5cc None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:30.201219 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:30.215586 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc74c7-1d60-4116-960a-86fbf40064d4 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:30.278177 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01e1c4-dc91-497a-b0a6-d02e654ae5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:30.278877 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01e1c4-dc91-497a-b0a6-d02e654ae5f2 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:30.279254 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01e1c4-dc91-497a-b0a6-d02e654ae5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:30.279552 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01e1c4-dc91-497a-b0a6-d02e654ae5f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:30.292177 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff01e1c4-dc91-497a-b0a6-d02e654ae5f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:30.300869 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-ff01e1c4-dc91-497a-b0a6-d02e654ae5f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 23:01:30.300869 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:30.300869 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:30.300869 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:30.300869 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:30.300869 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:30.300869 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:30.300869 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:30.300869 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:30.300869 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:30.300869 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:30.300869 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:30.300869 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:30.300869 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:30.300869 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:30.300869 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:30.302551 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:30.302551 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:30.302551 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:30.302551 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:30.302551 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:30.302551 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:30.302551 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:30.302551 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:30.302551 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:30.302551 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:30.302551 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:30.302551 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:30.302551 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:30.302551 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:30.302551 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:30.302551 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:30.305289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:30.305289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:30.305289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:30.305289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:30.305289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:30.305289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:30.305289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:30.305289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:30.305289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:30.305289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:30.305289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 23:01:30.305289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:30.305289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 163/343] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:01:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:30.305289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01e1c4-dc91-497a-b0a6-d02e654ae5f2 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:30.380377 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:30.416803 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01e1c4-dc91-497a-b0a6-d02e654ae5f2 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:30.453174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43a2bc3-a727-417e-96be-00f1a0c97422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:30.453996 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a2bc3-a727-417e-96be-00f1a0c97422 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:30.454276 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a2bc3-a727-417e-96be-00f1a0c97422 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:30.455263 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a2bc3-a727-417e-96be-00f1a0c97422 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:30.467106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43a2bc3-a727-417e-96be-00f1a0c97422 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:30.475559 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-a43a2bc3-a727-417e-96be-00f1a0c97422 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 23:01:30.475559 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:30.475559 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:30.475559 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:30.475559 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:30.475559 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:30.475559 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:30.475559 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:30.475559 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:30.475559 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:30.475559 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:30.475559 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:30.475559 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:30.475559 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:30.475559 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:30.475559 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:30.477296 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:30.477296 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:30.477296 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:30.477296 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:30.477296 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:30.477296 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:30.477296 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:30.477296 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:30.477296 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:30.477296 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:30.477296 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:30.477296 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:30.477296 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:30.477296 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:30.477296 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:30.477296 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:30.479472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:30.479472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:30.479472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:30.479472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:30.479472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:30.479472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:30.479472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:30.479472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:30.479472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:30.479472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:30.479472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 23:01:30.479472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:30.479472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 164/344] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jul 13 23:01:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:30.479472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43a2bc3-a727-417e-96be-00f1a0c97422 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:30.513527 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f59268-2945-4b01-9e64-e4335447c5cc None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:30.584379 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b364ce1-9732-40c0-bc24-ace225d34aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:30.584954 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b364ce1-9732-40c0-bc24-ace225d34aec None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:30.585240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b364ce1-9732-40c0-bc24-ace225d34aec None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:30.585521 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b364ce1-9732-40c0-bc24-ace225d34aec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:30.589051 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43a2bc3-a727-417e-96be-00f1a0c97422 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:30.612141 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:30.627210 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945da706-3b28-4af7-a392-b2f9aac0c30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:30.629390 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945da706-3b28-4af7-a392-b2f9aac0c30f None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:30.629885 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945da706-3b28-4af7-a392-b2f9aac0c30f None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:30.630346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945da706-3b28-4af7-a392-b2f9aac0c30f None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:30.631295 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945da706-3b28-4af7-a392-b2f9aac0c30f None admin] RBAC: Adding query filter params (name=member) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:30.631937 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945da706-3b28-4af7-a392-b2f9aac0c30f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:30.633559 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945da706-3b28-4af7-a392-b2f9aac0c30f None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:30.643415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 165/345] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jul 13 23:01:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:30.644605 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945da706-3b28-4af7-a392-b2f9aac0c30f None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:30.696113 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b364ce1-9732-40c0-bc24-ace225d34aec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:30.697918 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b364ce1-9732-40c0-bc24-ace225d34aec None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:30.706248 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:30.765007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945da706-3b28-4af7-a392-b2f9aac0c30f None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:30.801427 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947b567-d5f2-42dd-9ab3-ff37ab65d902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:30.802102 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947b567-d5f2-42dd-9ab3-ff37ab65d902 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:30.802382 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947b567-d5f2-42dd-9ab3-ff37ab65d902 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:30.802654 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947b567-d5f2-42dd-9ab3-ff37ab65d902 None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:30.803407 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7947b567-d5f2-42dd-9ab3-ff37ab65d902 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:30.804785 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7947b567-d5f2-42dd-9ab3-ff37ab65d902 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:30.806262 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7947b567-d5f2-42dd-9ab3-ff37ab65d902 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:30.815083 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 166/346] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:01:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:30.820230 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947b567-d5f2-42dd-9ab3-ff37ab65d902 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:30.834950 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 181/347] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:01:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 658 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 23:01:30.836220 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05203ff-2c18-49f2-b681-c4c39df49bfa None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:30.836633 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05203ff-2c18-49f2-b681-c4c39df49bfa None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:30.836915 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05203ff-2c18-49f2-b681-c4c39df49bfa None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:30.842919 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 182/348] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:30.843948 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05203ff-2c18-49f2-b681-c4c39df49bfa None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:30.857822 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:30.949485 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947b567-d5f2-42dd-9ab3-ff37ab65d902 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:30.976200 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05203ff-2c18-49f2-b681-c4c39df49bfa None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:30.986777 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:30.998621 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe84e6-7f89-44a1-928e-df7253dc23f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:30.999911 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe84e6-7f89-44a1-928e-df7253dc23f5 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:31.000795 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe84e6-7f89-44a1-928e-df7253dc23f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:31.001257 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe84e6-7f89-44a1-928e-df7253dc23f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:31.031339 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2643ba2-38d1-4273-b895-1932e42a0486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:31.032023 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2643ba2-38d1-4273-b895-1932e42a0486 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:31.032303 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2643ba2-38d1-4273-b895-1932e42a0486 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:31.032574 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2643ba2-38d1-4273-b895-1932e42a0486 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:31.108036 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe84e6-7f89-44a1-928e-df7253dc23f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:31.110640 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe84e6-7f89-44a1-928e-df7253dc23f5 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:31.128174 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2643ba2-38d1-4273-b895-1932e42a0486 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:31.130442 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2643ba2-38d1-4273-b895-1932e42a0486 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:31.146052 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:31.251382 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 167/349] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:01:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 23:01:31.251382 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe84e6-7f89-44a1-928e-df7253dc23f5 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:31.286762 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 183/350] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:01:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 23:01:31.290743 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cdd500-cbc6-452c-a826-13bf24d7c1a9 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:31.291050 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cdd500-cbc6-452c-a826-13bf24d7c1a9 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:31.291332 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cdd500-cbc6-452c-a826-13bf24d7c1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:31.307627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:31.320480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c8cdd500-cbc6-452c-a826-13bf24d7c1a9 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:31.320720 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c8cdd500-cbc6-452c-a826-13bf24d7c1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:31.419926 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 184/351] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:31.421450 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fbe4f2-b10d-4249-90f4-7c4cc54ee43d None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:31.421740 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fbe4f2-b10d-4249-90f4-7c4cc54ee43d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:31.422016 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fbe4f2-b10d-4249-90f4-7c4cc54ee43d None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:31.423402 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 185/352] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:31.425217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fbe4f2-b10d-4249-90f4-7c4cc54ee43d None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:31.448599 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe84e6-7f89-44a1-928e-df7253dc23f5 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:31.497132 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:31.505944 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ddb058-639f-4ff2-9afd-302da117fd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:31.508615 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ddb058-639f-4ff2-9afd-302da117fd76 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:31.508615 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ddb058-639f-4ff2-9afd-302da117fd76 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:31.508615 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ddb058-639f-4ff2-9afd-302da117fd76 None admin] PATH_INFO: `/v3/projects/bd646bf2ad574160a5fa8742f70da34c` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:31.510428 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ddb058-639f-4ff2-9afd-302da117fd76 None admin] RBAC: Authorizing `identity:get_project(project_id=bd646bf2ad574160a5fa8742f70da34c)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:31.512153 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ddb058-639f-4ff2-9afd-302da117fd76 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:31.513929 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 168/353] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:01:31 2022] GET /identity/v3/projects/bd646bf2ad574160a5fa8742f70da34c => generated 298 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:31.514703 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ddb058-639f-4ff2-9afd-302da117fd76 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:31.585255 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fbe4f2-b10d-4249-90f4-7c4cc54ee43d None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:31.633621 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bf2a55-6973-4b33-9bb7-0edf3d10476e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:31.635348 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf2a55-6973-4b33-9bb7-0edf3d10476e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:31.635854 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf2a55-6973-4b33-9bb7-0edf3d10476e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:31.636318 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bf2a55-6973-4b33-9bb7-0edf3d10476e None admin] PATH_INFO: `/v3/projects/b64d10f5c7424d909a79d57916eaeac5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:31.641118 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bf2a55-6973-4b33-9bb7-0edf3d10476e None admin] RBAC: Authorizing `identity:get_project(project_id=b64d10f5c7424d909a79d57916eaeac5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:31.643630 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bf2a55-6973-4b33-9bb7-0edf3d10476e None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:31.645736 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 186/354] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:01:31 2022] GET /identity/v3/projects/b64d10f5c7424d909a79d57916eaeac5 => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:31.646993 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea5874f-f0b6-4ba0-8816-118422c48533 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:31.651714 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea5874f-f0b6-4ba0-8816-118422c48533 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:31.651714 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea5874f-f0b6-4ba0-8816-118422c48533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:31.659032 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:31.675259 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-9ea5874f-f0b6-4ba0-8816-118422c48533 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:31.677311 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9ea5874f-f0b6-4ba0-8816-118422c48533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:31.677471 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ddb058-639f-4ff2-9afd-302da117fd76 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:31.729576 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3913ceab-5a5a-4739-9ffd-faa8302f03b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:31.730572 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3913ceab-5a5a-4739-9ffd-faa8302f03b2 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:31.730859 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3913ceab-5a5a-4739-9ffd-faa8302f03b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:31.731273 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3913ceab-5a5a-4739-9ffd-faa8302f03b2 None admin] PATH_INFO: `/v3/projects/bd646bf2ad574160a5fa8742f70da34c` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:31.732576 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3913ceab-5a5a-4739-9ffd-faa8302f03b2 None admin] RBAC: Authorizing `identity:get_project(project_id=bd646bf2ad574160a5fa8742f70da34c)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:31.734063 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3913ceab-5a5a-4739-9ffd-faa8302f03b2 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:31.736147 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 169/355] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:01:31 2022] GET /identity/v3/projects/bd646bf2ad574160a5fa8742f70da34c => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:31.740999 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3913ceab-5a5a-4739-9ffd-faa8302f03b2 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:31.778165 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 187/356] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:31.780351 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16501d-0a81-4f97-8f9e-aff8b1ae60f7 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:31.780680 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16501d-0a81-4f97-8f9e-aff8b1ae60f7 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:31.780960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16501d-0a81-4f97-8f9e-aff8b1ae60f7 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:31.782352 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 188/357] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:31.783040 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d16501d-0a81-4f97-8f9e-aff8b1ae60f7 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:31.819325 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:31.863156 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3913ceab-5a5a-4739-9ffd-faa8302f03b2 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:31.912489 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007bec6e-1330-43f2-83cc-50e5ece78655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:31.913244 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007bec6e-1330-43f2-83cc-50e5ece78655 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:31.913577 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007bec6e-1330-43f2-83cc-50e5ece78655 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:31.913866 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007bec6e-1330-43f2-83cc-50e5ece78655 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:31.919524 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d16501d-0a81-4f97-8f9e-aff8b1ae60f7 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:31.926927 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007bec6e-1330-43f2-83cc-50e5ece78655 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:31.936007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-007bec6e-1330-43f2-83cc-50e5ece78655 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 23:01:31.936007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:31.936007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:31.936007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:31.936007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:31.936007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:31.936007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:31.936007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:31.936007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:31.936007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:31.936007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:31.936007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:31.936007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:31.936007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:31.936007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:31.936007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:31.936965 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:31.936965 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:31.936965 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:31.936965 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:31.936965 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:31.936965 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:31.936965 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:31.936965 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:31.936965 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:31.936965 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:31.936965 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:31.936965 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:31.936965 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:31.936965 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:31.936965 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:31.936965 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:31.937740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:31.937740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:31.937740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:31.937740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:31.937740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:31.937740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:31.937740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:31.937740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:31.937740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:31.937740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:31.937740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 23:01:31.937740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:31.937740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 170/358] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jul 13 23:01:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:31.938407 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007bec6e-1330-43f2-83cc-50e5ece78655 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:31.969551 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41f04e-429a-48f7-a1c9-d893bb686ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:31.970407 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41f04e-429a-48f7-a1c9-d893bb686ffc None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:31.970690 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41f04e-429a-48f7-a1c9-d893bb686ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:31.970970 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41f04e-429a-48f7-a1c9-d893bb686ffc None admin] PATH_INFO: `/v3/groups/a4b41296b17f4bed8c47ea51f9a06baf` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:31.973815 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:31.980719 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d41f04e-429a-48f7-a1c9-d893bb686ffc None admin] RBAC: Authorizing `identity:get_group(group_id=a4b41296b17f4bed8c47ea51f9a06baf)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:31.983363 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d41f04e-429a-48f7-a1c9-d893bb686ffc None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:31.985595 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 189/359] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Jul 13 23:01:31 2022] GET /identity/v3/groups/a4b41296b17f4bed8c47ea51f9a06baf => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:31.986450 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41f04e-429a-48f7-a1c9-d893bb686ffc None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:32.073685 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007bec6e-1330-43f2-83cc-50e5ece78655 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:32.103827 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:32.114765 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086a8715-0d86-4d55-a39c-ad4d9eab2807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:32.115658 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a8715-0d86-4d55-a39c-ad4d9eab2807 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:32.115944 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a8715-0d86-4d55-a39c-ad4d9eab2807 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:32.116241 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a8715-0d86-4d55-a39c-ad4d9eab2807 None admin] PATH_INFO: `/v3/groups/a4b41296b17f4bed8c47ea51f9a06baf` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:32.117439 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086a8715-0d86-4d55-a39c-ad4d9eab2807 None admin] RBAC: Authorizing `identity:get_group(group_id=a4b41296b17f4bed8c47ea51f9a06baf)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:32.119378 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086a8715-0d86-4d55-a39c-ad4d9eab2807 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:32.121922 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 171/360] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Jul 13 23:01:31 2022] GET /identity/v3/groups/a4b41296b17f4bed8c47ea51f9a06baf => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:32.127467 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41f04e-429a-48f7-a1c9-d893bb686ffc None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:32.128317 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b902b-15fd-44dd-8d9e-b51ff464bd31 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:32.128606 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b902b-15fd-44dd-8d9e-b51ff464bd31 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:32.128887 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b902b-15fd-44dd-8d9e-b51ff464bd31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:32.149978 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-2f0b902b-15fd-44dd-8d9e-b51ff464bd31 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:32.151215 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-2f0b902b-15fd-44dd-8d9e-b51ff464bd31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:32.171987 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d344dece-ed09-44a9-be46-7de6dbbfe421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:32.172766 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d344dece-ed09-44a9-be46-7de6dbbfe421 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:32.173042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d344dece-ed09-44a9-be46-7de6dbbfe421 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:32.173318 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d344dece-ed09-44a9-be46-7de6dbbfe421 None admin] PATH_INFO: `/v3/groups/a4b41296b17f4bed8c47ea51f9a06baf` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:32.174639 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d344dece-ed09-44a9-be46-7de6dbbfe421 None admin] RBAC: Authorizing `identity:get_group(group_id=a4b41296b17f4bed8c47ea51f9a06baf)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:32.176130 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d344dece-ed09-44a9-be46-7de6dbbfe421 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:32.177751 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 190/361] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Jul 13 23:01:31 2022] GET /identity/v3/groups/a4b41296b17f4bed8c47ea51f9a06baf => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:32.178520 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d344dece-ed09-44a9-be46-7de6dbbfe421 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:32.235368 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 172/362] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:32.239927 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b902b-15fd-44dd-8d9e-b51ff464bd31 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:32.255902 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:32.302271 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d344dece-ed09-44a9-be46-7de6dbbfe421 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:32.346518 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54612a1-8d8d-4bbb-a2fd-9f841c84ac94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:32.347335 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54612a1-8d8d-4bbb-a2fd-9f841c84ac94 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:32.347631 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54612a1-8d8d-4bbb-a2fd-9f841c84ac94 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:32.347903 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54612a1-8d8d-4bbb-a2fd-9f841c84ac94 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:32.363929 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54612a1-8d8d-4bbb-a2fd-9f841c84ac94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:32.373792 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-b54612a1-8d8d-4bbb-a2fd-9f841c84ac94 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:01:32.373792 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:32.373792 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:32.373792 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:32.373792 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:32.373792 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:32.373792 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:32.373792 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:32.373792 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:32.373792 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:32.373792 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:32.373792 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:32.373792 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:32.373792 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:32.373792 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:32.373792 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:32.374813 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:32.374813 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:32.374813 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:32.374813 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:32.374813 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:32.374813 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:32.374813 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:32.374813 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:32.374813 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:32.374813 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:32.374813 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:32.374813 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:32.374813 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:32.374813 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:32.374813 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:32.374813 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:32.375739 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:32.375739 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:32.375739 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:32.375739 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:32.375739 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:32.375739 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:32.375739 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:32.375739 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:32.375739 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:32.375739 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:32.375739 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:01:32.375739 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:32.375739 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 191/363] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jul 13 23:01:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:32.376391 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54612a1-8d8d-4bbb-a2fd-9f841c84ac94 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:32.387694 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b902b-15fd-44dd-8d9e-b51ff464bd31 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:32.416345 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:32.430286 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddadb43-5709-45a2-8308-bf1d24684f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:32.431814 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddadb43-5709-45a2-8308-bf1d24684f19 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:32.432252 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddadb43-5709-45a2-8308-bf1d24684f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:32.432538 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddadb43-5709-45a2-8308-bf1d24684f19 None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:32.433316 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddadb43-5709-45a2-8308-bf1d24684f19 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:32.433612 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddadb43-5709-45a2-8308-bf1d24684f19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:32.435011 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddadb43-5709-45a2-8308-bf1d24684f19 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:32.445782 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 173/364] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:01:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:32.448159 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddadb43-5709-45a2-8308-bf1d24684f19 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:32.500654 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54612a1-8d8d-4bbb-a2fd-9f841c84ac94 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:32.534677 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:32.545006 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe7f7f-99a2-48bb-a27f-b6d8fecf5de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:32.545685 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe7f7f-99a2-48bb-a27f-b6d8fecf5de1 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:32.545961 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe7f7f-99a2-48bb-a27f-b6d8fecf5de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:32.546245 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe7f7f-99a2-48bb-a27f-b6d8fecf5de1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:32.547089 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfe7f7f-99a2-48bb-a27f-b6d8fecf5de1 None admin] RBAC: Adding query filter params (group.id=a4b41296b17f4bed8c47ea51f9a06baf, role.id=f005bdf39a0743238b3476f6b88b3684, scope.project.id=bd646bf2ad574160a5fa8742f70da34c) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:32.547400 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfe7f7f-99a2-48bb-a27f-b6d8fecf5de1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:32.548972 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfe7f7f-99a2-48bb-a27f-b6d8fecf5de1 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:32.555625 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 192/365] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Jul 13 23:01:32 2022] GET /identity/v3/role_assignments?group.id=a4b41296b17f4bed8c47ea51f9a06baf&scope.project.id=bd646bf2ad574160a5fa8742f70da34c&role.id=f005bdf39a0743238b3476f6b88b3684 => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:32.557271 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe7f7f-99a2-48bb-a27f-b6d8fecf5de1 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:32.571968 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddadb43-5709-45a2-8308-bf1d24684f19 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:32.616776 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcf0ffd-a360-40fc-a64d-85bd709b4365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:32.617456 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf0ffd-a360-40fc-a64d-85bd709b4365 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:32.617740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf0ffd-a360-40fc-a64d-85bd709b4365 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:32.618017 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcf0ffd-a360-40fc-a64d-85bd709b4365 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:32.619732 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcf0ffd-a360-40fc-a64d-85bd709b4365 None admin] RBAC: Adding query filter params (group.id=a4b41296b17f4bed8c47ea51f9a06baf, role.id=f005bdf39a0743238b3476f6b88b3684, scope.project.id=b64d10f5c7424d909a79d57916eaeac5) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:32.619909 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcf0ffd-a360-40fc-a64d-85bd709b4365 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:32.621434 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcf0ffd-a360-40fc-a64d-85bd709b4365 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:32.633211 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 174/366] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Jul 13 23:01:32 2022] GET /identity/v3/role_assignments?group.id=a4b41296b17f4bed8c47ea51f9a06baf&scope.project.id=b64d10f5c7424d909a79d57916eaeac5&role.id=f005bdf39a0743238b3476f6b88b3684 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:32.634069 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcf0ffd-a360-40fc-a64d-85bd709b4365 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:32.712718 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe7f7f-99a2-48bb-a27f-b6d8fecf5de1 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:32.735571 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:32.764748 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a71000-740b-484a-894d-25ab07840227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:32.765813 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a71000-740b-484a-894d-25ab07840227 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:32.766293 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a71000-740b-484a-894d-25ab07840227 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:32.766783 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a71000-740b-484a-894d-25ab07840227 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:32.767902 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a71000-740b-484a-894d-25ab07840227 None admin] RBAC: Adding query filter params (group.id=a4b41296b17f4bed8c47ea51f9a06baf, role.id=5f0ca8371a394d339621748b570e78ee, scope.project.id=bd646bf2ad574160a5fa8742f70da34c) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:32.768425 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a71000-740b-484a-894d-25ab07840227 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:32.770131 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a71000-740b-484a-894d-25ab07840227 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:32.777268 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 193/367] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Jul 13 23:01:32 2022] GET /identity/v3/role_assignments?group.id=a4b41296b17f4bed8c47ea51f9a06baf&scope.project.id=bd646bf2ad574160a5fa8742f70da34c&role.id=5f0ca8371a394d339621748b570e78ee => generated 260 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:32.778118 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a71000-740b-484a-894d-25ab07840227 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:32.808461 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcf0ffd-a360-40fc-a64d-85bd709b4365 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:32.854047 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c1b561-a4c4-4f4d-8ed0-73dc84087550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:32.855883 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1b561-a4c4-4f4d-8ed0-73dc84087550 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:32.856474 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1b561-a4c4-4f4d-8ed0-73dc84087550 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:32.856955 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1b561-a4c4-4f4d-8ed0-73dc84087550 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:32.870282 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c1b561-a4c4-4f4d-8ed0-73dc84087550 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:32.878770 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-44c1b561-a4c4-4f4d-8ed0-73dc84087550 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 23:01:32.878770 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:32.878770 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:32.878770 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:32.878770 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:32.878770 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:32.878770 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:32.878770 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:32.878770 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:32.878770 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:32.878770 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:32.878770 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:32.878770 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:32.878770 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:32.878770 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:32.878770 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:32.879796 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:32.879796 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:32.879796 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:32.879796 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:32.879796 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:32.879796 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:32.879796 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:32.879796 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:32.879796 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:32.879796 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:32.879796 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:32.879796 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:32.879796 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:32.879796 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:32.879796 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:32.879796 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:32.880586 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:32.880586 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:32.880586 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:32.880586 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:32.880586 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:32.880586 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:32.880586 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:32.880586 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:32.880586 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:32.880586 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:32.880586 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 23:01:32.880586 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:32.884505 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 175/368] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:01:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:32.885688 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c1b561-a4c4-4f4d-8ed0-73dc84087550 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:32.887972 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:32.904701 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a71000-740b-484a-894d-25ab07840227 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:33.004479 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c1b561-a4c4-4f4d-8ed0-73dc84087550 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:33.021142 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:33.045464 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7d2e73-d254-4b24-b7cc-e8859beca768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:33.046083 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4704b4-6cc7-44f5-b025-bdd9fa0f9576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:33.046214 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d2e73-d254-4b24-b7cc-e8859beca768 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:33.046415 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d2e73-d254-4b24-b7cc-e8859beca768 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:33.046688 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d2e73-d254-4b24-b7cc-e8859beca768 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:33.047444 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7d2e73-d254-4b24-b7cc-e8859beca768 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:33.047721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4704b4-6cc7-44f5-b025-bdd9fa0f9576 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:33.047821 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7d2e73-d254-4b24-b7cc-e8859beca768 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:33.048284 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4704b4-6cc7-44f5-b025-bdd9fa0f9576 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:33.048759 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4704b4-6cc7-44f5-b025-bdd9fa0f9576 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:33.049210 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7d2e73-d254-4b24-b7cc-e8859beca768 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:33.078355 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 194/369] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:01:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:33.079179 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7d2e73-d254-4b24-b7cc-e8859beca768 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:33.130195 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4704b4-6cc7-44f5-b025-bdd9fa0f9576 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:33.132094 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4704b4-6cc7-44f5-b025-bdd9fa0f9576 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:33.272261 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 176/370] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:01:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 23:01:33.273702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4704b4-6cc7-44f5-b025-bdd9fa0f9576 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:33.287904 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:33.313559 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7d2e73-d254-4b24-b7cc-e8859beca768 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:33.373419 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:33.406051 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4704b4-6cc7-44f5-b025-bdd9fa0f9576 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:33.444651 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07690c-e5e2-4111-b1ee-33d0894f5527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:33.445334 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07690c-e5e2-4111-b1ee-33d0894f5527 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:33.445621 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07690c-e5e2-4111-b1ee-33d0894f5527 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:33.445899 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07690c-e5e2-4111-b1ee-33d0894f5527 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:33.451325 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f85cc0-6a63-4845-9f23-bb1b3d20733a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:33.452034 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f85cc0-6a63-4845-9f23-bb1b3d20733a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:33.452323 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f85cc0-6a63-4845-9f23-bb1b3d20733a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:33.452598 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f85cc0-6a63-4845-9f23-bb1b3d20733a None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:33.453365 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f85cc0-6a63-4845-9f23-bb1b3d20733a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:33.453671 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f85cc0-6a63-4845-9f23-bb1b3d20733a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:33.455064 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f85cc0-6a63-4845-9f23-bb1b3d20733a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:33.475177 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 195/371] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jul 13 23:01:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:33.476127 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f85cc0-6a63-4845-9f23-bb1b3d20733a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:33.528371 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d07690c-e5e2-4111-b1ee-33d0894f5527 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:33.530046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d07690c-e5e2-4111-b1ee-33d0894f5527 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:33.545071 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:33.647261 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 177/372] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:01:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 23:01:33.648150 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07690c-e5e2-4111-b1ee-33d0894f5527 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:33.656557 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f85cc0-6a63-4845-9f23-bb1b3d20733a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:33.712931 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f550f-0104-4d34-80f1-a76ce3c723e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:33.713886 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f550f-0104-4d34-80f1-a76ce3c723e9 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:33.714174 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f550f-0104-4d34-80f1-a76ce3c723e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:33.714452 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f550f-0104-4d34-80f1-a76ce3c723e9 None admin] PATH_INFO: `/v3/projects/b64d10f5c7424d909a79d57916eaeac5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:33.715840 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31f550f-0104-4d34-80f1-a76ce3c723e9 None admin] RBAC: Authorizing `identity:get_project(project_id=b64d10f5c7424d909a79d57916eaeac5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:33.717344 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31f550f-0104-4d34-80f1-a76ce3c723e9 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:33.719173 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 196/373] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:01:33 2022] GET /identity/v3/projects/b64d10f5c7424d909a79d57916eaeac5 => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:33.724498 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f550f-0104-4d34-80f1-a76ce3c723e9 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:33.762138 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07690c-e5e2-4111-b1ee-33d0894f5527 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:33.775049 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:33.775938 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:33.804265 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbac49-9a25-4727-939e-ec0aa5bfdabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:33.805092 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbac49-9a25-4727-939e-ec0aa5bfdabf None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:33.805377 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbac49-9a25-4727-939e-ec0aa5bfdabf None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:33.805698 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbac49-9a25-4727-939e-ec0aa5bfdabf None admin] PATH_INFO: `/v3/users/0a98f73270bd4ce699b3f0bbd6c9f9c0` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:33.807091 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fbac49-9a25-4727-939e-ec0aa5bfdabf None admin] RBAC: Authorizing `identity:get_user(user_id=0a98f73270bd4ce699b3f0bbd6c9f9c0)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:33.808605 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fbac49-9a25-4727-939e-ec0aa5bfdabf None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:33.810837 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 178/374] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:01:33 2022] GET /identity/v3/users/0a98f73270bd4ce699b3f0bbd6c9f9c0 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:33.811734 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbac49-9a25-4727-939e-ec0aa5bfdabf None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:33.852037 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f550f-0104-4d34-80f1-a76ce3c723e9 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:33.924441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbac49-9a25-4727-939e-ec0aa5bfdabf None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:33.956801 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a72de90-da9e-46ee-a00a-57cc2b082752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:33.958028 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72de90-da9e-46ee-a00a-57cc2b082752 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:33.958452 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72de90-da9e-46ee-a00a-57cc2b082752 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:33.958866 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72de90-da9e-46ee-a00a-57cc2b082752 None admin] PATH_INFO: `/v3/projects/c5ffbcb80ba6476698f912924febd384` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:33.962292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f55a3f6-c6bb-4579-8873-19ee04e78164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:33.963280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a72de90-da9e-46ee-a00a-57cc2b082752 None admin] RBAC: Authorizing `identity:get_project(project_id=c5ffbcb80ba6476698f912924febd384)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:33.963422 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55a3f6-c6bb-4579-8873-19ee04e78164 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:33.963812 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55a3f6-c6bb-4579-8873-19ee04e78164 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:33.964188 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55a3f6-c6bb-4579-8873-19ee04e78164 None admin] PATH_INFO: `/v3/groups/a4b41296b17f4bed8c47ea51f9a06baf` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:33.965375 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a72de90-da9e-46ee-a00a-57cc2b082752 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:33.965768 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f55a3f6-c6bb-4579-8873-19ee04e78164 None admin] RBAC: Authorizing `identity:get_group(group_id=a4b41296b17f4bed8c47ea51f9a06baf)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:33.967365 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f55a3f6-c6bb-4579-8873-19ee04e78164 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:33.968303 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 197/375] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:01:33 2022] GET /identity/v3/projects/c5ffbcb80ba6476698f912924febd384 => generated 344 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:33.969313 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a72de90-da9e-46ee-a00a-57cc2b082752 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:33.969837 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 179/376] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Jul 13 23:01:33 2022] GET /identity/v3/groups/a4b41296b17f4bed8c47ea51f9a06baf => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:33.975604 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f55a3f6-c6bb-4579-8873-19ee04e78164 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:34.077888 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:34.166799 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a72de90-da9e-46ee-a00a-57cc2b082752 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:34.183943 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:34.193715 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f55a3f6-c6bb-4579-8873-19ee04e78164 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:34.205122 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e93c41-0e34-4ac9-98e3-25ca82875f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:34.205827 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e93c41-0e34-4ac9-98e3-25ca82875f42 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:34.206115 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e93c41-0e34-4ac9-98e3-25ca82875f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:34.206397 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e93c41-0e34-4ac9-98e3-25ca82875f42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:34.207231 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e93c41-0e34-4ac9-98e3-25ca82875f42 None admin] RBAC: Adding query filter params (role.id=2562dfc9fd0845678cb5beff6a715255, user.id=0a98f73270bd4ce699b3f0bbd6c9f9c0, scope.system=all) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:34.207561 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e93c41-0e34-4ac9-98e3-25ca82875f42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:34.209612 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e93c41-0e34-4ac9-98e3-25ca82875f42 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:34.223562 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 198/377] 10.222.0.60 () {56 vars in 1308 bytes} [Wed Jul 13 23:01:33 2022] GET /identity/v3/role_assignments?user.id=0a98f73270bd4ce699b3f0bbd6c9f9c0&scope.system=all&role.id=2562dfc9fd0845678cb5beff6a715255 => generated 510 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:34.224492 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e93c41-0e34-4ac9-98e3-25ca82875f42 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:34.347950 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb0cf1b-882c-427f-85c8-17a80ff377d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:34.348813 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb0cf1b-882c-427f-85c8-17a80ff377d7 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:34.349100 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb0cf1b-882c-427f-85c8-17a80ff377d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:34.349375 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb0cf1b-882c-427f-85c8-17a80ff377d7 None admin] PATH_INFO: `/v3/groups/149aa4b606fb4c1395ceb432e543c411` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:34.382368 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb0cf1b-882c-427f-85c8-17a80ff377d7 None admin] RBAC: Authorizing `identity:get_group(group_id=149aa4b606fb4c1395ceb432e543c411)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:34.383962 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb0cf1b-882c-427f-85c8-17a80ff377d7 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:34.386156 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 180/378] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Jul 13 23:01:33 2022] GET /identity/v3/groups/149aa4b606fb4c1395ceb432e543c411 => generated 229 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:34.394530 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e93c41-0e34-4ac9-98e3-25ca82875f42 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:34.395296 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb0cf1b-882c-427f-85c8-17a80ff377d7 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:34.467960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661423c-15af-42a2-be44-1c0326a36986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:34.468998 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661423c-15af-42a2-be44-1c0326a36986 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:34.469475 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661423c-15af-42a2-be44-1c0326a36986 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:34.470703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661423c-15af-42a2-be44-1c0326a36986 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:34.471764 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8661423c-15af-42a2-be44-1c0326a36986 None admin] RBAC: Adding query filter params (group.id=a4b41296b17f4bed8c47ea51f9a06baf, role.id=5f0ca8371a394d339621748b570e78ee, scope.project.id=b64d10f5c7424d909a79d57916eaeac5) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:34.476481 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8661423c-15af-42a2-be44-1c0326a36986 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:34.478839 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8661423c-15af-42a2-be44-1c0326a36986 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:34.489397 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 199/379] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Jul 13 23:01:34 2022] GET /identity/v3/role_assignments?group.id=a4b41296b17f4bed8c47ea51f9a06baf&scope.project.id=b64d10f5c7424d909a79d57916eaeac5&role.id=5f0ca8371a394d339621748b570e78ee => generated 260 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:34.582438 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb0cf1b-882c-427f-85c8-17a80ff377d7 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:34.598084 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:34.632760 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe6fbbe-d6f1-47b3-8626-27d6fb1b745f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:34.633449 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe6fbbe-d6f1-47b3-8626-27d6fb1b745f None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:34.634251 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe6fbbe-d6f1-47b3-8626-27d6fb1b745f None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:34.634543 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe6fbbe-d6f1-47b3-8626-27d6fb1b745f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:34.635432 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe6fbbe-d6f1-47b3-8626-27d6fb1b745f None admin] RBAC: Adding query filter params (group.id=149aa4b606fb4c1395ceb432e543c411, role.id=8cfd0b6e71e14e0983c3eb4d7478e027, scope.project.id=c5ffbcb80ba6476698f912924febd384) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:34.635774 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe6fbbe-d6f1-47b3-8626-27d6fb1b745f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:34.637288 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe6fbbe-d6f1-47b3-8626-27d6fb1b745f None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:34.648573 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 181/380] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Jul 13 23:01:34 2022] GET /identity/v3/role_assignments?group.id=149aa4b606fb4c1395ceb432e543c411&scope.project.id=c5ffbcb80ba6476698f912924febd384&role.id=8cfd0b6e71e14e0983c3eb4d7478e027 => generated 260 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:35.204557 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62699cab-658f-4f6b-9aa9-c8a1878c7090 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:35.205199 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62699cab-658f-4f6b-9aa9-c8a1878c7090 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:35.205742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62699cab-658f-4f6b-9aa9-c8a1878c7090 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:35.207654 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 200/381] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:35.213160 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5439d-0336-4b6b-8d65-24ed0581555d None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:35.213731 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5439d-0336-4b6b-8d65-24ed0581555d None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:35.214196 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5439d-0336-4b6b-8d65-24ed0581555d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:35.232748 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3527dfc-5f4f-4fbd-9e87-f0b14d999e33 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:35.233056 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3527dfc-5f4f-4fbd-9e87-f0b14d999e33 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:35.233314 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3527dfc-5f4f-4fbd-9e87-f0b14d999e33 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:35.234953 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 201/382] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:35.240203 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-8ef5439d-0336-4b6b-8d65-24ed0581555d None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:35.241205 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac299957-7f03-4b37-b380-29e79ee7af94 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:35.241496 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac299957-7f03-4b37-b380-29e79ee7af94 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:35.241776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac299957-7f03-4b37-b380-29e79ee7af94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:35.242342 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-8ef5439d-0336-4b6b-8d65-24ed0581555d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:35.275515 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-ac299957-7f03-4b37-b380-29e79ee7af94 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:35.278087 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ac299957-7f03-4b37-b380-29e79ee7af94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:35.337092 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:35.369559 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 182/383] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:35.370918 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8369b62-2c52-4d52-9bc6-c8672ae655cc None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:35.371207 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8369b62-2c52-4d52-9bc6-c8672ae655cc None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:35.371489 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8369b62-2c52-4d52-9bc6-c8672ae655cc None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:35.377136 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 183/384] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:35.382499 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e1b3f-cc7d-453c-bc30-ef621c5954e4 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:35.382792 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e1b3f-cc7d-453c-bc30-ef621c5954e4 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:35.383079 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e1b3f-cc7d-453c-bc30-ef621c5954e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:35.393752 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 202/385] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:35.404771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-125e1b3f-cc7d-453c-bc30-ef621c5954e4 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:35.406032 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-125e1b3f-cc7d-453c-bc30-ef621c5954e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:35.433888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac299957-7f03-4b37-b380-29e79ee7af94 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:35.500629 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 184/386] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:35.502452 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125e1b3f-cc7d-453c-bc30-ef621c5954e4 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:35.580024 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:35.604666 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac299957-7f03-4b37-b380-29e79ee7af94 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:35.652375 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce1073-eebb-4e26-9ad0-cd64c6b01adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:35.653760 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce1073-eebb-4e26-9ad0-cd64c6b01adf None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:35.654199 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce1073-eebb-4e26-9ad0-cd64c6b01adf None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:35.654909 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce1073-eebb-4e26-9ad0-cd64c6b01adf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:35.666825 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125e1b3f-cc7d-453c-bc30-ef621c5954e4 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:35.668755 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fce1073-eebb-4e26-9ad0-cd64c6b01adf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:35.676635 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-7fce1073-eebb-4e26-9ad0-cd64c6b01adf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 23:01:35.676635 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:35.676635 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:35.676635 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:35.676635 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:35.676635 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:35.676635 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:35.676635 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:35.676635 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:35.676635 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:35.676635 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:35.676635 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:35.676635 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:35.676635 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:35.676635 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:35.676635 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:35.677770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:35.677770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:35.677770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:35.677770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:35.677770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:35.677770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:35.677770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:35.677770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:35.677770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:35.677770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:35.677770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:35.677770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:35.677770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:35.677770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:35.677770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:35.677770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:35.678631 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:35.678631 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:35.678631 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:35.678631 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:35.678631 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:35.678631 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:35.678631 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:35.678631 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:35.678631 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:35.678631 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:35.678631 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 23:01:35.678631 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:35.678631 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 203/387] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:01:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:01:35.679276 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce1073-eebb-4e26-9ad0-cd64c6b01adf None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:35.709257 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:35.714853 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9aded9-4a92-41b1-8b48-336fa3138c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:35.716505 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9aded9-4a92-41b1-8b48-336fa3138c4b None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:35.717255 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9aded9-4a92-41b1-8b48-336fa3138c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:35.718002 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9aded9-4a92-41b1-8b48-336fa3138c4b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:35.737805 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9aded9-4a92-41b1-8b48-336fa3138c4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:35.753613 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-bb9aded9-4a92-41b1-8b48-336fa3138c4b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 23:01:35.753613 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:35.753613 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:35.753613 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:35.753613 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:35.753613 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:35.753613 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:35.753613 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:35.753613 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:35.753613 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:35.753613 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:35.753613 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:35.753613 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:35.753613 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:35.753613 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:35.753613 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:35.754525 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:35.754525 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:35.754525 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:35.754525 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:35.754525 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:35.754525 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:35.754525 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:35.754525 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:35.754525 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:35.754525 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:35.754525 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:35.754525 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:35.754525 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:35.754525 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:35.754525 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:35.754525 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:35.755289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:35.755289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:35.755289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:35.755289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:35.755289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:35.755289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:35.755289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:35.755289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:35.755289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:35.755289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:35.755289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 23:01:35.755289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:35.758046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 185/388] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:01:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:35.759744 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9aded9-4a92-41b1-8b48-336fa3138c4b None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:35.799234 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce1073-eebb-4e26-9ad0-cd64c6b01adf None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:35.835816 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d4d9c-61ec-49f1-ad1d-41da5fc0ed81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:35.836999 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d4d9c-61ec-49f1-ad1d-41da5fc0ed81 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:35.837464 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d4d9c-61ec-49f1-ad1d-41da5fc0ed81 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:35.837909 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d4d9c-61ec-49f1-ad1d-41da5fc0ed81 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:35.849206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d4d9c-61ec-49f1-ad1d-41da5fc0ed81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:35.860959 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:35.862390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-9e3d4d9c-61ec-49f1-ad1d-41da5fc0ed81 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 23:01:35.862390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:35.862390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:35.862390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:35.862390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:35.862390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:35.862390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:35.862390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:35.862390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:35.862390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:35.862390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:35.862390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:35.862390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:35.862390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:35.862390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:35.862390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:35.864150 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:35.864150 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:35.864150 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:35.864150 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:35.864150 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:35.864150 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:35.864150 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:35.864150 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:35.864150 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:35.864150 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:35.864150 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:35.864150 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:35.864150 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:35.864150 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:35.864150 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:35.864150 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:35.865738 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:35.865738 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:35.865738 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:35.865738 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:35.865738 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:35.865738 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:35.865738 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:35.865738 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:35.865738 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:35.865738 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:35.865738 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 23:01:35.865738 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:35.870081 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9aded9-4a92-41b1-8b48-336fa3138c4b None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:35.870610 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 204/389] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jul 13 23:01:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:35.872442 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d4d9c-61ec-49f1-ad1d-41da5fc0ed81 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:35.911066 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb80b8d5-8c9e-47b1-b456-cf4b07bec823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:35.911771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80b8d5-8c9e-47b1-b456-cf4b07bec823 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:35.912076 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80b8d5-8c9e-47b1-b456-cf4b07bec823 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:35.912338 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80b8d5-8c9e-47b1-b456-cf4b07bec823 None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:35.913093 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb80b8d5-8c9e-47b1-b456-cf4b07bec823 None admin] RBAC: Adding query filter params (name=member) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:35.913409 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb80b8d5-8c9e-47b1-b456-cf4b07bec823 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:35.914813 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb80b8d5-8c9e-47b1-b456-cf4b07bec823 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:35.923463 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 186/390] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jul 13 23:01:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:35.924290 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb80b8d5-8c9e-47b1-b456-cf4b07bec823 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:36.029878 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb80b8d5-8c9e-47b1-b456-cf4b07bec823 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:36.032350 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:36.067451 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d4d9c-61ec-49f1-ad1d-41da5fc0ed81 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:36.069404 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d7c8dd-70a8-4714-be6e-cb123e9b45f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:36.070085 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7c8dd-70a8-4714-be6e-cb123e9b45f9 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:36.070370 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7c8dd-70a8-4714-be6e-cb123e9b45f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:36.070643 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d7c8dd-70a8-4714-be6e-cb123e9b45f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:36.071441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d7c8dd-70a8-4714-be6e-cb123e9b45f9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:36.071778 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d7c8dd-70a8-4714-be6e-cb123e9b45f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:36.073505 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d7c8dd-70a8-4714-be6e-cb123e9b45f9 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:36.082123 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 187/391] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:01:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:36.082929 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d7c8dd-70a8-4714-be6e-cb123e9b45f9 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:36.107769 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:36.124842 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b64eb-300f-43ca-b0c3-1d002fabab3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:36.125754 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b64eb-300f-43ca-b0c3-1d002fabab3e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:36.126178 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b64eb-300f-43ca-b0c3-1d002fabab3e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:36.126590 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b64eb-300f-43ca-b0c3-1d002fabab3e None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:36.127481 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011b64eb-300f-43ca-b0c3-1d002fabab3e None admin] RBAC: Adding query filter params (name=member) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:36.127997 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011b64eb-300f-43ca-b0c3-1d002fabab3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:36.129522 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011b64eb-300f-43ca-b0c3-1d002fabab3e None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:36.138796 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 205/392] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jul 13 23:01:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:36.139841 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b64eb-300f-43ca-b0c3-1d002fabab3e None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:36.206234 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d7c8dd-70a8-4714-be6e-cb123e9b45f9 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:36.242781 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b64eb-300f-43ca-b0c3-1d002fabab3e None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:36.251983 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c39f1d-0ba2-4a5a-8d4b-584631e5e1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:36.252907 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c39f1d-0ba2-4a5a-8d4b-584631e5e1ec None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:36.253190 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c39f1d-0ba2-4a5a-8d4b-584631e5e1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:36.253465 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c39f1d-0ba2-4a5a-8d4b-584631e5e1ec None admin] PATH_INFO: `/v3/groups/a4b41296b17f4bed8c47ea51f9a06baf` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:36.256994 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c39f1d-0ba2-4a5a-8d4b-584631e5e1ec None admin] RBAC: Authorizing `identity:get_group(group_id=a4b41296b17f4bed8c47ea51f9a06baf)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:36.259637 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c39f1d-0ba2-4a5a-8d4b-584631e5e1ec None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:36.262925 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 188/393] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Jul 13 23:01:36 2022] GET /identity/v3/groups/a4b41296b17f4bed8c47ea51f9a06baf => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:36.263992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c39f1d-0ba2-4a5a-8d4b-584631e5e1ec None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:36.286803 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdbfb6f-20ea-4e1d-bd13-32ff8b60f076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:36.287609 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdbfb6f-20ea-4e1d-bd13-32ff8b60f076 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:36.287896 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdbfb6f-20ea-4e1d-bd13-32ff8b60f076 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:36.288171 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdbfb6f-20ea-4e1d-bd13-32ff8b60f076 None admin] PATH_INFO: `/v3/groups/a4b41296b17f4bed8c47ea51f9a06baf` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:36.289369 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdbfb6f-20ea-4e1d-bd13-32ff8b60f076 None admin] RBAC: Authorizing `identity:get_group(group_id=a4b41296b17f4bed8c47ea51f9a06baf)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:36.291353 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdbfb6f-20ea-4e1d-bd13-32ff8b60f076 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:36.293115 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 206/394] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Jul 13 23:01:36 2022] GET /identity/v3/groups/a4b41296b17f4bed8c47ea51f9a06baf => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:36.293890 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdbfb6f-20ea-4e1d-bd13-32ff8b60f076 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:36.348093 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:36.384214 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:36.400917 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c39f1d-0ba2-4a5a-8d4b-584631e5e1ec None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:36.426914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdbfb6f-20ea-4e1d-bd13-32ff8b60f076 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:36.445825 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf00d3-1587-41e2-986f-0b39092827ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:36.446993 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf00d3-1587-41e2-986f-0b39092827ba None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:36.448240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf00d3-1587-41e2-986f-0b39092827ba None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:36.448735 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf00d3-1587-41e2-986f-0b39092827ba None admin] PATH_INFO: `/v3/groups/a4b41296b17f4bed8c47ea51f9a06baf` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:36.450767 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bf00d3-1587-41e2-986f-0b39092827ba None admin] RBAC: Authorizing `identity:get_group(group_id=a4b41296b17f4bed8c47ea51f9a06baf)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:36.453193 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bf00d3-1587-41e2-986f-0b39092827ba None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:36.456345 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 189/395] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Jul 13 23:01:36 2022] GET /identity/v3/groups/a4b41296b17f4bed8c47ea51f9a06baf => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:36.457997 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf00d3-1587-41e2-986f-0b39092827ba None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:36.463893 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934fdf5-6108-40c4-a6bf-4e39a3d3337b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:36.464404 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934fdf5-6108-40c4-a6bf-4e39a3d3337b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:36.464690 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934fdf5-6108-40c4-a6bf-4e39a3d3337b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:36.465014 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934fdf5-6108-40c4-a6bf-4e39a3d3337b None admin] PATH_INFO: `/v3/projects/b64d10f5c7424d909a79d57916eaeac5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:36.466251 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2934fdf5-6108-40c4-a6bf-4e39a3d3337b None admin] RBAC: Authorizing `identity:get_project(project_id=b64d10f5c7424d909a79d57916eaeac5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:36.467753 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2934fdf5-6108-40c4-a6bf-4e39a3d3337b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:36.474308 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 207/396] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:01:36 2022] GET /identity/v3/projects/b64d10f5c7424d909a79d57916eaeac5 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:36.475164 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934fdf5-6108-40c4-a6bf-4e39a3d3337b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:36.573697 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf00d3-1587-41e2-986f-0b39092827ba None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:36.587428 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:36.611473 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934fdf5-6108-40c4-a6bf-4e39a3d3337b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:36.617793 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0cfddd-07b0-4c54-8f8c-7682231d887d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:36.618710 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0cfddd-07b0-4c54-8f8c-7682231d887d None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:36.618994 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0cfddd-07b0-4c54-8f8c-7682231d887d None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:36.619271 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0cfddd-07b0-4c54-8f8c-7682231d887d None admin] PATH_INFO: `/v3/projects/bd646bf2ad574160a5fa8742f70da34c` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:36.620522 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0cfddd-07b0-4c54-8f8c-7682231d887d None admin] RBAC: Authorizing `identity:get_project(project_id=bd646bf2ad574160a5fa8742f70da34c)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:36.622012 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0cfddd-07b0-4c54-8f8c-7682231d887d None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:36.624091 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 190/397] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:01:36 2022] GET /identity/v3/projects/bd646bf2ad574160a5fa8742f70da34c => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:36.625029 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0cfddd-07b0-4c54-8f8c-7682231d887d None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:36.656822 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:36.657680 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf5935-13e8-482b-8f1b-ddc1006358d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:36.659418 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf5935-13e8-482b-8f1b-ddc1006358d0 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:36.659862 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf5935-13e8-482b-8f1b-ddc1006358d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:36.660196 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf5935-13e8-482b-8f1b-ddc1006358d0 None admin] PATH_INFO: `/v3/projects/bd646bf2ad574160a5fa8742f70da34c` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:36.662928 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf5935-13e8-482b-8f1b-ddc1006358d0 None admin] RBAC: Authorizing `identity:get_project(project_id=bd646bf2ad574160a5fa8742f70da34c)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:36.664537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf5935-13e8-482b-8f1b-ddc1006358d0 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:36.667838 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 208/398] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:01:36 2022] GET /identity/v3/projects/bd646bf2ad574160a5fa8742f70da34c => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:36.669160 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf5935-13e8-482b-8f1b-ddc1006358d0 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:36.743068 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0cfddd-07b0-4c54-8f8c-7682231d887d None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:36.782094 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e39198-3f97-4981-bbee-bb7778e8bbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:36.783345 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e39198-3f97-4981-bbee-bb7778e8bbaf None admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:36.783878 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e39198-3f97-4981-bbee-bb7778e8bbaf None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:36.784369 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e39198-3f97-4981-bbee-bb7778e8bbaf None admin] PATH_INFO: `/v3/projects/b64d10f5c7424d909a79d57916eaeac5/groups/a4b41296b17f4bed8c47ea51f9a06baf/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:36.786709 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e39198-3f97-4981-bbee-bb7778e8bbaf None admin] RBAC: Authorizing `identity:create_grant(project_id=b64d10f5c7424d909a79d57916eaeac5, group_id=a4b41296b17f4bed8c47ea51f9a06baf, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:36.788528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e39198-3f97-4981-bbee-bb7778e8bbaf None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:36.806225 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 191/399] 10.222.0.60 () {58 vars in 1442 bytes} [Wed Jul 13 23:01:36 2022] PUT /identity/v3/projects/b64d10f5c7424d909a79d57916eaeac5/groups/a4b41296b17f4bed8c47ea51f9a06baf/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:01:36.807554 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e39198-3f97-4981-bbee-bb7778e8bbaf None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:36.830359 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf5935-13e8-482b-8f1b-ddc1006358d0 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:36.886266 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c88b8-2212-4e64-9821-a393520d0519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:36.887237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c88b8-2212-4e64-9821-a393520d0519 None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:36.887498 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c88b8-2212-4e64-9821-a393520d0519 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:36.887801 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c88b8-2212-4e64-9821-a393520d0519 None admin] PATH_INFO: `/v3/projects/bd646bf2ad574160a5fa8742f70da34c/groups/a4b41296b17f4bed8c47ea51f9a06baf/roles/5f0ca8371a394d339621748b570e78ee` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:36.890408 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221c88b8-2212-4e64-9821-a393520d0519 None admin] RBAC: Authorizing `identity:create_grant(project_id=bd646bf2ad574160a5fa8742f70da34c, group_id=a4b41296b17f4bed8c47ea51f9a06baf, role_id=5f0ca8371a394d339621748b570e78ee)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:36.892105 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221c88b8-2212-4e64-9821-a393520d0519 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:36.909811 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 209/400] 10.222.0.60 () {58 vars in 1442 bytes} [Wed Jul 13 23:01:36 2022] PUT /identity/v3/projects/bd646bf2ad574160a5fa8742f70da34c/groups/a4b41296b17f4bed8c47ea51f9a06baf/roles/5f0ca8371a394d339621748b570e78ee => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:01:36.916227 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:36.935784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e39198-3f97-4981-bbee-bb7778e8bbaf None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:36.978018 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3c4f5-807a-4d89-97ea-758686c9b264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:36.978978 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3c4f5-807a-4d89-97ea-758686c9b264 None admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:36.979272 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3c4f5-807a-4d89-97ea-758686c9b264 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:36.979564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3c4f5-807a-4d89-97ea-758686c9b264 None admin] PATH_INFO: `/v3/projects/bd646bf2ad574160a5fa8742f70da34c/groups/a4b41296b17f4bed8c47ea51f9a06baf/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:36.981931 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a3c4f5-807a-4d89-97ea-758686c9b264 None admin] RBAC: Authorizing `identity:create_grant(project_id=bd646bf2ad574160a5fa8742f70da34c, group_id=a4b41296b17f4bed8c47ea51f9a06baf, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:36.983571 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a3c4f5-807a-4d89-97ea-758686c9b264 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:36.998601 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 192/401] 10.222.0.60 () {58 vars in 1442 bytes} [Wed Jul 13 23:01:36 2022] PUT /identity/v3/projects/bd646bf2ad574160a5fa8742f70da34c/groups/a4b41296b17f4bed8c47ea51f9a06baf/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:01:37.292959 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059ccb3-6be7-447b-ba5a-cd550cd7eeab None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:37.293429 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059ccb3-6be7-447b-ba5a-cd550cd7eeab None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:37.293521 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1059ccb3-6be7-447b-ba5a-cd550cd7eeab None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:37.295103 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 210/402] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:37.302313 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156a8ef-3e1a-4d0c-9653-3fc0333e683e None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:37.303027 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156a8ef-3e1a-4d0c-9653-3fc0333e683e None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:37.303620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156a8ef-3e1a-4d0c-9653-3fc0333e683e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:37.336861 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-d156a8ef-3e1a-4d0c-9653-3fc0333e683e None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:37.338795 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-d156a8ef-3e1a-4d0c-9653-3fc0333e683e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:37.458089 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 193/403] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 23:01:37.481510 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059ccb3-6be7-447b-ba5a-cd550cd7eeab None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:37.512030 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:37.570847 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4201c27-2996-41a7-b54b-beca536e8611 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:37.571158 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4201c27-2996-41a7-b54b-beca536e8611 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:37.571389 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4201c27-2996-41a7-b54b-beca536e8611 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:37.574422 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 194/404] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:01:37.578750 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25891676-5693-44c5-a578-4972e932f92d None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:37.579038 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25891676-5693-44c5-a578-4972e932f92d None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:37.579313 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25891676-5693-44c5-a578-4972e932f92d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:37.596455 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-25891676-5693-44c5-a578-4972e932f92d None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:37.597726 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-25891676-5693-44c5-a578-4972e932f92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:37.677129 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1059ccb3-6be7-447b-ba5a-cd550cd7eeab None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:37.716519 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a778e5-4439-4d6e-9f9e-601ec671a160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:37.717426 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a778e5-4439-4d6e-9f9e-601ec671a160 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:37.717751 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a778e5-4439-4d6e-9f9e-601ec671a160 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:37.718025 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a778e5-4439-4d6e-9f9e-601ec671a160 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:37.730906 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a778e5-4439-4d6e-9f9e-601ec671a160 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:37.739215 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-93a778e5-4439-4d6e-9f9e-601ec671a160 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:01:37.739215 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:37.739215 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:37.739215 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:37.739215 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:37.739215 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:37.739215 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:37.739215 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:37.739215 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:37.739215 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:37.739215 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:37.739215 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:37.739215 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:37.739215 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:37.739215 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:37.739215 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:37.740251 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:37.740251 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:37.740251 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:37.740251 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:37.740251 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:37.740251 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:37.740251 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:37.740251 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:37.740251 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:37.740251 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:37.740251 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:37.740251 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:37.740251 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:37.740251 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:37.740251 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:37.740251 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:37.741071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:37.741071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:37.741071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:37.741071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:37.741071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:37.741071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:37.741071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:37.741071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:37.741071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:37.741071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:37.741071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:01:37.741071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:37.741692 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 195/406] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 23:01:37.745621 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 211/406] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jul 13 23:01:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:37.748043 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25891676-5693-44c5-a578-4972e932f92d None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:37.752132 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:37.768888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a778e5-4439-4d6e-9f9e-601ec671a160 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:37.846493 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:37.878735 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a778e5-4439-4d6e-9f9e-601ec671a160 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:37.914159 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620d9be-b8b8-4f3f-a9a3-19499741e4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:37.915009 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620d9be-b8b8-4f3f-a9a3-19499741e4b4 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:37.915291 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620d9be-b8b8-4f3f-a9a3-19499741e4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:37.915583 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620d9be-b8b8-4f3f-a9a3-19499741e4b4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:37.926993 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620d9be-b8b8-4f3f-a9a3-19499741e4b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:37.935914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-1620d9be-b8b8-4f3f-a9a3-19499741e4b4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 23:01:37.935914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:37.935914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:37.935914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:37.935914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:37.935914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:37.935914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:37.935914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:37.935914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:37.935914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:37.935914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:37.935914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:37.935914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:37.935914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:37.935914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:37.935914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:37.936960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:37.936960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:37.936960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:37.936960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:37.936960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:37.936960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:37.936960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:37.936960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:37.936960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:37.936960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:37.936960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:37.936960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:37.936960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:37.936960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:37.936960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:37.936960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:37.937730 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:37.937730 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:37.937730 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:37.937730 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:37.937730 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:37.937730 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:37.937730 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:37.937730 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:37.937730 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:37.937730 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:37.937730 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 23:01:37.937730 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:37.942154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 212/407] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jul 13 23:01:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:01:37.943462 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620d9be-b8b8-4f3f-a9a3-19499741e4b4 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:37.958198 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25891676-5693-44c5-a578-4972e932f92d None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:38.047135 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb3c28a-169d-4a42-853d-97a1563e512c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:38.047856 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3c28a-169d-4a42-853d-97a1563e512c None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:38.048141 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3c28a-169d-4a42-853d-97a1563e512c None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:38.048451 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3c28a-169d-4a42-853d-97a1563e512c None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:38.049371 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb3c28a-169d-4a42-853d-97a1563e512c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:38.049797 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb3c28a-169d-4a42-853d-97a1563e512c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:38.051308 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb3c28a-169d-4a42-853d-97a1563e512c None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:38.052661 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620d9be-b8b8-4f3f-a9a3-19499741e4b4 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:38.076068 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 196/408] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:01:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:38.091989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb3c28a-169d-4a42-853d-97a1563e512c None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:38.094543 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:38.103858 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf693d-3450-4a68-8f59-72066d0c4069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:38.104464 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf693d-3450-4a68-8f59-72066d0c4069 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:38.104763 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf693d-3450-4a68-8f59-72066d0c4069 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:38.105057 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf693d-3450-4a68-8f59-72066d0c4069 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:38.105910 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bf693d-3450-4a68-8f59-72066d0c4069 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:38.106263 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bf693d-3450-4a68-8f59-72066d0c4069 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:38.107892 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bf693d-3450-4a68-8f59-72066d0c4069 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:38.124099 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 213/409] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:01:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:01:38.135133 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf693d-3450-4a68-8f59-72066d0c4069 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:38.195925 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:38.238477 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb3c28a-169d-4a42-853d-97a1563e512c None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:38.247475 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf693d-3450-4a68-8f59-72066d0c4069 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:38.279313 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb4336-b20b-44de-a27d-c41ed975f7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:38.281992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb4336-b20b-44de-a27d-c41ed975f7e2 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:38.283280 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb4336-b20b-44de-a27d-c41ed975f7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:38.284509 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb4336-b20b-44de-a27d-c41ed975f7e2 None admin] PATH_INFO: `/v3/groups/149aa4b606fb4c1395ceb432e543c411` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:38.286193 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cb4336-b20b-44de-a27d-c41ed975f7e2 None admin] RBAC: Authorizing `identity:get_group(group_id=149aa4b606fb4c1395ceb432e543c411)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:38.289038 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cb4336-b20b-44de-a27d-c41ed975f7e2 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:38.292754 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 197/410] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Jul 13 23:01:38 2022] GET /identity/v3/groups/149aa4b606fb4c1395ceb432e543c411 => generated 229 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:38.298767 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb4336-b20b-44de-a27d-c41ed975f7e2 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:38.304010 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3182a427-af84-4ae1-8064-b55d8560f4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:38.304669 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182a427-af84-4ae1-8064-b55d8560f4de None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:38.304951 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182a427-af84-4ae1-8064-b55d8560f4de None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:38.305222 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182a427-af84-4ae1-8064-b55d8560f4de None admin] PATH_INFO: `/v3/groups/a4b41296b17f4bed8c47ea51f9a06baf` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:38.311100 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3182a427-af84-4ae1-8064-b55d8560f4de None admin] RBAC: Authorizing `identity:get_group(group_id=a4b41296b17f4bed8c47ea51f9a06baf)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:38.312642 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3182a427-af84-4ae1-8064-b55d8560f4de None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:38.315124 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 214/411] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Jul 13 23:01:38 2022] GET /identity/v3/groups/a4b41296b17f4bed8c47ea51f9a06baf => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:01:38.320943 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3182a427-af84-4ae1-8064-b55d8560f4de None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:38.425407 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb4336-b20b-44de-a27d-c41ed975f7e2 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:38.458079 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:38.463144 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:38.464562 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f8806-097d-4dc9-ab65-c2dac715b58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:38.465719 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f8806-097d-4dc9-ab65-c2dac715b58e None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:38.466234 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f8806-097d-4dc9-ab65-c2dac715b58e None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:38.466709 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f8806-097d-4dc9-ab65-c2dac715b58e None admin] PATH_INFO: `/v3/projects/c5ffbcb80ba6476698f912924febd384` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:38.468163 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f8806-097d-4dc9-ab65-c2dac715b58e None admin] RBAC: Authorizing `identity:get_project(project_id=c5ffbcb80ba6476698f912924febd384)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:38.469918 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f8806-097d-4dc9-ab65-c2dac715b58e None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:38.472257 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 198/412] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:01:38 2022] GET /identity/v3/projects/c5ffbcb80ba6476698f912924febd384 => generated 344 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:38.478163 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f8806-097d-4dc9-ab65-c2dac715b58e None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:38.547333 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3182a427-af84-4ae1-8064-b55d8560f4de None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:38.588526 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1756c8-d49f-48bc-ade6-29bd13ac390c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:38.589789 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1756c8-d49f-48bc-ade6-29bd13ac390c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:38.590275 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1756c8-d49f-48bc-ade6-29bd13ac390c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:38.590774 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1756c8-d49f-48bc-ade6-29bd13ac390c None admin] PATH_INFO: `/v3/projects/b64d10f5c7424d909a79d57916eaeac5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:38.592332 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1756c8-d49f-48bc-ade6-29bd13ac390c None admin] RBAC: Authorizing `identity:get_project(project_id=b64d10f5c7424d909a79d57916eaeac5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:38.594164 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1756c8-d49f-48bc-ade6-29bd13ac390c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:38.597184 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 215/413] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:01:38 2022] GET /identity/v3/projects/b64d10f5c7424d909a79d57916eaeac5 => generated 302 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:01:38.602844 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1756c8-d49f-48bc-ade6-29bd13ac390c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:38.642887 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f8806-097d-4dc9-ab65-c2dac715b58e None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:38.681119 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957cd313-7cae-40c2-9723-a0a398cd8a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:38.682374 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957cd313-7cae-40c2-9723-a0a398cd8a63 None admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:38.682858 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957cd313-7cae-40c2-9723-a0a398cd8a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:38.683355 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957cd313-7cae-40c2-9723-a0a398cd8a63 None admin] PATH_INFO: `/v3/projects/c5ffbcb80ba6476698f912924febd384/groups/149aa4b606fb4c1395ceb432e543c411/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:38.685631 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957cd313-7cae-40c2-9723-a0a398cd8a63 None admin] RBAC: Authorizing `identity:create_grant(project_id=c5ffbcb80ba6476698f912924febd384, group_id=149aa4b606fb4c1395ceb432e543c411, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:38.687399 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957cd313-7cae-40c2-9723-a0a398cd8a63 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:38.702738 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 199/414] 10.222.0.60 () {58 vars in 1442 bytes} [Wed Jul 13 23:01:38 2022] PUT /identity/v3/projects/c5ffbcb80ba6476698f912924febd384/groups/149aa4b606fb4c1395ceb432e543c411/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:01:38.814568 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1756c8-d49f-48bc-ade6-29bd13ac390c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:38.858063 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:38.864903 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf278f-c14c-4b5b-affe-6de26389e8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:38.865856 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf278f-c14c-4b5b-affe-6de26389e8ce None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:38.866142 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf278f-c14c-4b5b-affe-6de26389e8ce None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:38.866426 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf278f-c14c-4b5b-affe-6de26389e8ce None admin] PATH_INFO: `/v3/projects/b64d10f5c7424d909a79d57916eaeac5/groups/a4b41296b17f4bed8c47ea51f9a06baf/roles/5f0ca8371a394d339621748b570e78ee` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:38.873800 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf278f-c14c-4b5b-affe-6de26389e8ce None admin] RBAC: Authorizing `identity:create_grant(project_id=b64d10f5c7424d909a79d57916eaeac5, group_id=a4b41296b17f4bed8c47ea51f9a06baf, role_id=5f0ca8371a394d339621748b570e78ee)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:38.876524 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf278f-c14c-4b5b-affe-6de26389e8ce None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:38.892188 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 216/415] 10.222.0.60 () {58 vars in 1442 bytes} [Wed Jul 13 23:01:38 2022] PUT /identity/v3/projects/b64d10f5c7424d909a79d57916eaeac5/groups/a4b41296b17f4bed8c47ea51f9a06baf/roles/5f0ca8371a394d339621748b570e78ee => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:01:39.625406 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f0e23d-dce7-48b8-9f24-4bbcac7559eb None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:39.626433 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f0e23d-dce7-48b8-9f24-4bbcac7559eb None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:39.626919 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f0e23d-dce7-48b8-9f24-4bbcac7559eb None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:39.628864 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 200/416] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:39.641271 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6094ee-c626-47fa-894a-87b72adabe8d None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:39.641771 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6094ee-c626-47fa-894a-87b72adabe8d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:39.641845 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6094ee-c626-47fa-894a-87b72adabe8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:39.671454 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-4c6094ee-c626-47fa-894a-87b72adabe8d None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:39.672702 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4c6094ee-c626-47fa-894a-87b72adabe8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:39.770846 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 217/417] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:39.779417 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737a5d2-d6c0-4015-b598-1ced1a55d0d4 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:39.779759 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737a5d2-d6c0-4015-b598-1ced1a55d0d4 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:39.780042 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737a5d2-d6c0-4015-b598-1ced1a55d0d4 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:39.782688 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 201/418] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:39.787150 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe9abff-ff45-4a08-894c-d9280ec6b7ee None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:39.787436 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe9abff-ff45-4a08-894c-d9280ec6b7ee None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:39.787738 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe9abff-ff45-4a08-894c-d9280ec6b7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:39.811360 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-9fe9abff-ff45-4a08-894c-d9280ec6b7ee None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:39.811685 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737a5d2-d6c0-4015-b598-1ced1a55d0d4 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:39.812585 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9fe9abff-ff45-4a08-894c-d9280ec6b7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:39.933421 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 218/419] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:39.936985 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:39.977183 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe9abff-ff45-4a08-894c-d9280ec6b7ee None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:39.996489 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737a5d2-d6c0-4015-b598-1ced1a55d0d4 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:40.041298 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:40.054756 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a243fb-2cb9-4dc6-beae-33280d2c8ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:40.056605 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a243fb-2cb9-4dc6-beae-33280d2c8ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:40.057097 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a243fb-2cb9-4dc6-beae-33280d2c8ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:40.057752 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a243fb-2cb9-4dc6-beae-33280d2c8ab7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:40.075470 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a243fb-2cb9-4dc6-beae-33280d2c8ab7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:40.088540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-64a243fb-2cb9-4dc6-beae-33280d2c8ab7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 23:01:40.088540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:40.088540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:40.088540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:40.088540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:40.088540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:40.088540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:40.088540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:40.088540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:40.088540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:40.088540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:40.088540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:40.088540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:40.088540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:40.088540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:40.088540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:40.089672 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:40.089672 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:40.089672 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:40.089672 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:40.089672 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:40.089672 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:40.089672 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:40.089672 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:40.089672 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:40.089672 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:40.089672 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:40.089672 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:40.089672 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:40.089672 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:40.089672 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:40.089672 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:40.090621 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:40.090621 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:40.090621 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:40.090621 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:40.090621 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:40.090621 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:40.090621 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:40.090621 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:40.090621 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:40.090621 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:40.090621 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 23:01:40.090621 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:40.093971 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 202/420] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:01:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:40.096697 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a25de3-c2ad-418e-b3de-6e590fdc35d0 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:40.096990 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a25de3-c2ad-418e-b3de-6e590fdc35d0 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:40.097269 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a25de3-c2ad-418e-b3de-6e590fdc35d0 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:40.098666 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 203/421] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:40.102978 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a25de3-c2ad-418e-b3de-6e590fdc35d0 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:40.115765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe9abff-ff45-4a08-894c-d9280ec6b7ee None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:40.160773 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e14e4-4723-440d-a308-806a4d0ade47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:40.161756 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e14e4-4723-440d-a308-806a4d0ade47 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:40.162215 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e14e4-4723-440d-a308-806a4d0ade47 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:40.162508 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e14e4-4723-440d-a308-806a4d0ade47 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:40.177210 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7e14e4-4723-440d-a308-806a4d0ade47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:40.188331 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-df7e14e4-4723-440d-a308-806a4d0ade47 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 23:01:40.188331 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:40.188331 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:40.188331 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:40.188331 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:40.188331 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:40.188331 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:40.188331 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:40.188331 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:40.188331 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:40.188331 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:40.188331 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:40.188331 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:40.188331 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:40.188331 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:40.188331 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:40.189267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:40.189267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:40.189267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:40.189267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:40.189267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:40.189267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:40.189267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:40.189267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:40.189267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:40.189267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:40.189267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:40.189267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:40.189267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:40.189267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:40.189267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:40.189267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:40.190525 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:40.190525 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:40.190525 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:40.190525 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:40.190525 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:40.190525 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:40.190525 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:40.190525 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:40.190525 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:40.190525 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:40.190525 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 23:01:40.190525 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:40.194886 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 219/422] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:01:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:40.198206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112536f0-195e-4a83-a43e-79a2b3a140d2 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:40.198500 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112536f0-195e-4a83-a43e-79a2b3a140d2 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:40.198766 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112536f0-195e-4a83-a43e-79a2b3a140d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:40.225963 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-112536f0-195e-4a83-a43e-79a2b3a140d2 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:40.227303 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-112536f0-195e-4a83-a43e-79a2b3a140d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:40.264030 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a25de3-c2ad-418e-b3de-6e590fdc35d0 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:40.298196 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:40.331558 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 220/423] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:40.331558 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112536f0-195e-4a83-a43e-79a2b3a140d2 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:40.331808 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bf0f28-a52c-4a31-b47d-a82577e4ee29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:40.331808 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf0f28-a52c-4a31-b47d-a82577e4ee29 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:40.331808 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf0f28-a52c-4a31-b47d-a82577e4ee29 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:40.331808 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf0f28-a52c-4a31-b47d-a82577e4ee29 None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:40.332085 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bf0f28-a52c-4a31-b47d-a82577e4ee29 None admin] RBAC: Adding query filter params (name=member) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:40.332085 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bf0f28-a52c-4a31-b47d-a82577e4ee29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:40.334341 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bf0f28-a52c-4a31-b47d-a82577e4ee29 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:40.371583 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 204/424] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jul 13 23:01:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:40.373406 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bf0f28-a52c-4a31-b47d-a82577e4ee29 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:40.413035 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:40.488551 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bf0f28-a52c-4a31-b47d-a82577e4ee29 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:40.505878 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112536f0-195e-4a83-a43e-79a2b3a140d2 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:40.525249 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43243caa-5123-48ef-a634-9ca016249de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:40.525901 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43243caa-5123-48ef-a634-9ca016249de2 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:40.526182 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43243caa-5123-48ef-a634-9ca016249de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:40.526455 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43243caa-5123-48ef-a634-9ca016249de2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:40.560496 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6ded8-0ab9-46fd-a337-7d769390fa3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:40.561217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6ded8-0ab9-46fd-a337-7d769390fa3f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:40.561497 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6ded8-0ab9-46fd-a337-7d769390fa3f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:40.561865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6ded8-0ab9-46fd-a337-7d769390fa3f None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:40.562626 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af6ded8-0ab9-46fd-a337-7d769390fa3f None admin] RBAC: Adding query filter params (name=member) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:40.562931 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af6ded8-0ab9-46fd-a337-7d769390fa3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:40.564655 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af6ded8-0ab9-46fd-a337-7d769390fa3f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:40.580186 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 221/425] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jul 13 23:01:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:40.581726 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6ded8-0ab9-46fd-a337-7d769390fa3f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:40.587792 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:40.610997 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43243caa-5123-48ef-a634-9ca016249de2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:40.612993 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43243caa-5123-48ef-a634-9ca016249de2 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:40.717047 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6ded8-0ab9-46fd-a337-7d769390fa3f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:40.718723 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:40.726924 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 205/426] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:01:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 23:01:40.728932 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43243caa-5123-48ef-a634-9ca016249de2 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:40.761351 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c66fa-5984-4364-8e16-7177d5ce0c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:40.762192 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c66fa-5984-4364-8e16-7177d5ce0c3f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:40.762477 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c66fa-5984-4364-8e16-7177d5ce0c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:40.762752 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c66fa-5984-4364-8e16-7177d5ce0c3f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:40.775872 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42c66fa-5984-4364-8e16-7177d5ce0c3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:40.790932 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-f42c66fa-5984-4364-8e16-7177d5ce0c3f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 23:01:40.790932 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:40.790932 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:40.790932 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:40.790932 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:40.790932 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:40.790932 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:40.790932 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:40.790932 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:40.790932 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:40.790932 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:40.790932 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:40.790932 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:40.790932 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:40.790932 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:40.790932 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:40.792133 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:40.792133 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:40.792133 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:40.792133 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:40.792133 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:40.792133 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:40.792133 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:40.792133 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:40.792133 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:40.792133 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:40.792133 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:40.792133 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:40.792133 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:40.792133 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:40.792133 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:40.792133 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:40.792997 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:40.792997 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:40.792997 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:40.792997 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:40.792997 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:40.792997 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:40.792997 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:40.792997 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:40.792997 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:40.792997 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:40.792997 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 23:01:40.792997 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:40.793603 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 222/427] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jul 13 23:01:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:40.800571 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c66fa-5984-4364-8e16-7177d5ce0c3f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:40.825760 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:40.847467 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43243caa-5123-48ef-a634-9ca016249de2 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:40.886785 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0140f9e6-257d-4f7a-83a6-a09c76d32632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:40.887848 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0140f9e6-257d-4f7a-83a6-a09c76d32632 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:40.888314 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0140f9e6-257d-4f7a-83a6-a09c76d32632 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:40.888793 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0140f9e6-257d-4f7a-83a6-a09c76d32632 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:40.967348 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0140f9e6-257d-4f7a-83a6-a09c76d32632 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:40.967916 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c66fa-5984-4364-8e16-7177d5ce0c3f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:40.969360 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0140f9e6-257d-4f7a-83a6-a09c76d32632 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:41.027253 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b666c438-6a17-46f0-8a23-54743cc858c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:41.028586 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666c438-6a17-46f0-8a23-54743cc858c3 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:41.029023 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666c438-6a17-46f0-8a23-54743cc858c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:41.029443 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666c438-6a17-46f0-8a23-54743cc858c3 None admin] PATH_INFO: `/v3/projects/b64d10f5c7424d909a79d57916eaeac5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:41.031600 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b666c438-6a17-46f0-8a23-54743cc858c3 None admin] RBAC: Authorizing `identity:get_project(project_id=b64d10f5c7424d909a79d57916eaeac5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:41.034198 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b666c438-6a17-46f0-8a23-54743cc858c3 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:41.037872 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 223/428] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:01:40 2022] GET /identity/v3/projects/b64d10f5c7424d909a79d57916eaeac5 => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:41.039355 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b666c438-6a17-46f0-8a23-54743cc858c3 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:41.065278 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:41.093046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 206/429] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:01:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 23:01:41.094286 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0140f9e6-257d-4f7a-83a6-a09c76d32632 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:41.098237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:41.208342 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0140f9e6-257d-4f7a-83a6-a09c76d32632 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:41.225968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b666c438-6a17-46f0-8a23-54743cc858c3 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:41.250264 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a2e12-1d70-46b1-ba9c-c255c89c6d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:41.251086 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a2e12-1d70-46b1-ba9c-c255c89c6d5b None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:41.251368 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a2e12-1d70-46b1-ba9c-c255c89c6d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:41.251674 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a2e12-1d70-46b1-ba9c-c255c89c6d5b None admin] PATH_INFO: `/v3/groups/a4b41296b17f4bed8c47ea51f9a06baf` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:41.253478 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182a2e12-1d70-46b1-ba9c-c255c89c6d5b None admin] RBAC: Authorizing `identity:get_group(group_id=a4b41296b17f4bed8c47ea51f9a06baf)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:41.255128 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182a2e12-1d70-46b1-ba9c-c255c89c6d5b None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:41.257020 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 207/430] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Jul 13 23:01:41 2022] GET /identity/v3/groups/a4b41296b17f4bed8c47ea51f9a06baf => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:41.262326 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a2e12-1d70-46b1-ba9c-c255c89c6d5b None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:41.281693 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16696b3-2cf8-4169-a40f-d462cffcdaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:41.282297 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16696b3-2cf8-4169-a40f-d462cffcdaab None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:41.282580 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16696b3-2cf8-4169-a40f-d462cffcdaab None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:41.282871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16696b3-2cf8-4169-a40f-d462cffcdaab None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:41.283677 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16696b3-2cf8-4169-a40f-d462cffcdaab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:41.283984 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16696b3-2cf8-4169-a40f-d462cffcdaab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:41.285443 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16696b3-2cf8-4169-a40f-d462cffcdaab None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:41.296188 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 224/431] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:01:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:41.297449 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c713a74-6327-4ab2-8c0a-2f6ec5000936 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:41.297801 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c713a74-6327-4ab2-8c0a-2f6ec5000936 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:41.298145 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c713a74-6327-4ab2-8c0a-2f6ec5000936 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:41.299766 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 225/432] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:41.305219 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c713a74-6327-4ab2-8c0a-2f6ec5000936 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:41.338082 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:41.400222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a2e12-1d70-46b1-ba9c-c255c89c6d5b None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:41.410981 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:41.423199 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c713a74-6327-4ab2-8c0a-2f6ec5000936 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:41.436828 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62887e15-f78e-4f4a-9e2f-f78a013c0626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:41.437705 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62887e15-f78e-4f4a-9e2f-f78a013c0626 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:41.437983 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62887e15-f78e-4f4a-9e2f-f78a013c0626 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:41.438256 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62887e15-f78e-4f4a-9e2f-f78a013c0626 None admin] PATH_INFO: `/v3/projects/bd646bf2ad574160a5fa8742f70da34c` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:41.439546 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62887e15-f78e-4f4a-9e2f-f78a013c0626 None admin] RBAC: Authorizing `identity:get_project(project_id=bd646bf2ad574160a5fa8742f70da34c)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:41.441048 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62887e15-f78e-4f4a-9e2f-f78a013c0626 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:41.442802 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 208/433] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:01:41 2022] GET /identity/v3/projects/bd646bf2ad574160a5fa8742f70da34c => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:41.443984 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62887e15-f78e-4f4a-9e2f-f78a013c0626 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:41.471392 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80fb3be-8b51-49fb-ad5a-049c5f876e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:41.472464 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80fb3be-8b51-49fb-ad5a-049c5f876e37 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:41.472922 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80fb3be-8b51-49fb-ad5a-049c5f876e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:41.473379 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80fb3be-8b51-49fb-ad5a-049c5f876e37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:41.474754 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80fb3be-8b51-49fb-ad5a-049c5f876e37 None admin] RBAC: Adding query filter params (group.id=a4b41296b17f4bed8c47ea51f9a06baf, role.id=f005bdf39a0743238b3476f6b88b3684, scope.project.id=b64d10f5c7424d909a79d57916eaeac5) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:41.475143 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80fb3be-8b51-49fb-ad5a-049c5f876e37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:41.478141 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80fb3be-8b51-49fb-ad5a-049c5f876e37 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:41.486744 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 226/434] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Jul 13 23:01:41 2022] GET /identity/v3/role_assignments?group.id=a4b41296b17f4bed8c47ea51f9a06baf&scope.project.id=b64d10f5c7424d909a79d57916eaeac5&role.id=f005bdf39a0743238b3476f6b88b3684 => generated 611 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:41.488852 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00165d5b-d5ba-4cbd-b2f8-4e3e0bc328ff None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:41.489340 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00165d5b-d5ba-4cbd-b2f8-4e3e0bc328ff None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:41.489813 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00165d5b-d5ba-4cbd-b2f8-4e3e0bc328ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:41.511467 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-00165d5b-d5ba-4cbd-b2f8-4e3e0bc328ff None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:41.512765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-00165d5b-d5ba-4cbd-b2f8-4e3e0bc328ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:41.591094 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62887e15-f78e-4f4a-9e2f-f78a013c0626 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:41.611740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:41.619464 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 227/435] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:41.621309 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e7cea-7d12-4392-92df-a7c4880e7d55 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:41.621476 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e7cea-7d12-4392-92df-a7c4880e7d55 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:41.621752 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e7cea-7d12-4392-92df-a7c4880e7d55 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:41.624434 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 228/436] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:41.624434 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e7cea-7d12-4392-92df-a7c4880e7d55 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:41.650692 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e682a37f-62c8-4fa6-9b83-9d42bb04c07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:41.651747 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682a37f-62c8-4fa6-9b83-9d42bb04c07f None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:41.652170 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682a37f-62c8-4fa6-9b83-9d42bb04c07f None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:41.652619 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682a37f-62c8-4fa6-9b83-9d42bb04c07f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:41.721425 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:41.743340 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e7cea-7d12-4392-92df-a7c4880e7d55 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:41.754163 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e682a37f-62c8-4fa6-9b83-9d42bb04c07f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:41.756141 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e682a37f-62c8-4fa6-9b83-9d42bb04c07f None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:41.782967 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b0baa4-ea39-4ce8-924e-a61af780d3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:41.783855 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0baa4-ea39-4ce8-924e-a61af780d3c2 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:41.784131 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0baa4-ea39-4ce8-924e-a61af780d3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:41.784402 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0baa4-ea39-4ce8-924e-a61af780d3c2 None admin] PATH_INFO: `/v3/groups/a4b41296b17f4bed8c47ea51f9a06baf` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:41.785597 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b0baa4-ea39-4ce8-924e-a61af780d3c2 None admin] RBAC: Authorizing `identity:get_group(group_id=a4b41296b17f4bed8c47ea51f9a06baf)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:41.787035 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b0baa4-ea39-4ce8-924e-a61af780d3c2 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:41.788760 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 229/437] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Jul 13 23:01:41 2022] GET /identity/v3/groups/a4b41296b17f4bed8c47ea51f9a06baf => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:41.794354 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6edcd-a08a-4384-8d63-43c1663f2b40 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:41.794354 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6edcd-a08a-4384-8d63-43c1663f2b40 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:41.794354 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6edcd-a08a-4384-8d63-43c1663f2b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:41.807156 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-4fc6edcd-a08a-4384-8d63-43c1663f2b40 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:41.808410 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4fc6edcd-a08a-4384-8d63-43c1663f2b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:41.861231 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 209/438] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:01:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 23:01:41.862082 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e682a37f-62c8-4fa6-9b83-9d42bb04c07f None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:41.876058 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 230/439] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:41.876919 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc6edcd-a08a-4384-8d63-43c1663f2b40 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:41.880324 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:41.969509 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e682a37f-62c8-4fa6-9b83-9d42bb04c07f None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:41.970802 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:41.988722 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc6edcd-a08a-4384-8d63-43c1663f2b40 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:42.007045 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a659d19-56c6-4f65-aa9c-2ac0f5377470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:42.007891 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a659d19-56c6-4f65-aa9c-2ac0f5377470 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:42.008174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a659d19-56c6-4f65-aa9c-2ac0f5377470 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:42.008980 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a659d19-56c6-4f65-aa9c-2ac0f5377470 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:42.020317 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a659d19-56c6-4f65-aa9c-2ac0f5377470 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:42.026537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25dacfb-f339-4703-89ad-9e926f65b3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:42.027328 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25dacfb-f339-4703-89ad-9e926f65b3ac None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:42.027776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25dacfb-f339-4703-89ad-9e926f65b3ac None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:42.028161 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25dacfb-f339-4703-89ad-9e926f65b3ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:42.028790 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-0a659d19-56c6-4f65-aa9c-2ac0f5377470 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:01:42.028790 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:42.028790 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:42.028790 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:42.028790 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:42.028790 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:42.028790 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:42.028790 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:42.028790 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:42.028790 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:42.028790 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:42.028790 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:42.028790 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:42.028790 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:42.028790 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:42.028790 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:42.029753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:42.029753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:42.029753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:42.029753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:42.029753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:42.029753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:42.029753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:42.029753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:42.029753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:42.029753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:42.029753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:42.029753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:42.029753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:42.029753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:42.029753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:42.029753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:42.030550 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:42.030550 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:42.030550 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:42.030550 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:42.030550 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:42.030550 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:42.030550 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:42.030550 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:42.030550 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:42.030550 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:42.030550 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:01:42.030550 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:42.031157 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25dacfb-f339-4703-89ad-9e926f65b3ac None admin] RBAC: Adding query filter params (group.id=a4b41296b17f4bed8c47ea51f9a06baf, role.id=f005bdf39a0743238b3476f6b88b3684, scope.project.id=bd646bf2ad574160a5fa8742f70da34c) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:42.031157 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25dacfb-f339-4703-89ad-9e926f65b3ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:42.031279 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 210/440] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jul 13 23:01:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:42.031342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25dacfb-f339-4703-89ad-9e926f65b3ac None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:42.031413 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a659d19-56c6-4f65-aa9c-2ac0f5377470 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:42.037880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 231/441] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Jul 13 23:01:41 2022] GET /identity/v3/role_assignments?group.id=a4b41296b17f4bed8c47ea51f9a06baf&scope.project.id=bd646bf2ad574160a5fa8742f70da34c&role.id=f005bdf39a0743238b3476f6b88b3684 => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:42.038720 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25dacfb-f339-4703-89ad-9e926f65b3ac None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:42.142719 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:42.157592 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25dacfb-f339-4703-89ad-9e926f65b3ac None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:42.169663 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a659d19-56c6-4f65-aa9c-2ac0f5377470 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:42.198282 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d473aff-e62d-4b54-9d43-66173321d5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:42.199639 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d473aff-e62d-4b54-9d43-66173321d5d7 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:42.200108 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d473aff-e62d-4b54-9d43-66173321d5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:42.200567 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d473aff-e62d-4b54-9d43-66173321d5d7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:42.207433 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca759092-0ee5-4491-a4ce-89a08dc61e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:42.208344 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca759092-0ee5-4491-a4ce-89a08dc61e5b None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:42.208618 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca759092-0ee5-4491-a4ce-89a08dc61e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:42.208894 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca759092-0ee5-4491-a4ce-89a08dc61e5b None admin] PATH_INFO: `/v3/projects/bd646bf2ad574160a5fa8742f70da34c` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:42.210127 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca759092-0ee5-4491-a4ce-89a08dc61e5b None admin] RBAC: Authorizing `identity:get_project(project_id=bd646bf2ad574160a5fa8742f70da34c)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:42.212192 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca759092-0ee5-4491-a4ce-89a08dc61e5b None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:42.218526 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 211/442] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:01:42 2022] GET /identity/v3/projects/bd646bf2ad574160a5fa8742f70da34c => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:42.218526 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca759092-0ee5-4491-a4ce-89a08dc61e5b None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:42.220718 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d473aff-e62d-4b54-9d43-66173321d5d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:42.229130 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-5d473aff-e62d-4b54-9d43-66173321d5d7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 23:01:42.229130 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:42.229130 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:42.229130 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:42.229130 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:42.229130 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:42.229130 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:42.229130 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:42.229130 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:42.229130 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:42.229130 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:42.229130 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:42.229130 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:42.229130 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:42.229130 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:42.229130 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:42.230218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:42.230218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:42.230218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:42.230218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:42.230218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:42.230218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:42.230218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:42.230218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:42.230218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:42.230218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:42.230218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:42.230218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:42.230218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:42.230218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:42.230218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:42.230218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:42.230995 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:42.230995 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:42.230995 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:42.230995 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:42.230995 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:42.230995 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:42.230995 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:42.230995 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:42.230995 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:42.230995 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:42.230995 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 23:01:42.230995 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:42.231867 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 232/443] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jul 13 23:01:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:42.232595 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d473aff-e62d-4b54-9d43-66173321d5d7 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:42.245565 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:42.338056 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca759092-0ee5-4491-a4ce-89a08dc61e5b None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:42.363734 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d473aff-e62d-4b54-9d43-66173321d5d7 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:42.392481 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77cbdcf-d60f-4778-a5b8-16c5512f8fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:42.393228 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77cbdcf-d60f-4778-a5b8-16c5512f8fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:42.393816 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77cbdcf-d60f-4778-a5b8-16c5512f8fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:42.393816 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77cbdcf-d60f-4778-a5b8-16c5512f8fd9 None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:42.394570 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77cbdcf-d60f-4778-a5b8-16c5512f8fd9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:42.395132 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77cbdcf-d60f-4778-a5b8-16c5512f8fd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:42.397465 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77cbdcf-d60f-4778-a5b8-16c5512f8fd9 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:42.406762 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 212/444] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:01:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:42.408191 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77cbdcf-d60f-4778-a5b8-16c5512f8fd9 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:42.413007 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9e7a38-54cd-43a3-a425-760846f4283a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:42.413871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e7a38-54cd-43a3-a425-760846f4283a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:42.414146 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e7a38-54cd-43a3-a425-760846f4283a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:42.414413 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e7a38-54cd-43a3-a425-760846f4283a None admin] PATH_INFO: `/v3/groups/a4b41296b17f4bed8c47ea51f9a06baf` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:42.415675 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9e7a38-54cd-43a3-a425-760846f4283a None admin] RBAC: Authorizing `identity:get_group(group_id=a4b41296b17f4bed8c47ea51f9a06baf)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:42.417154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9e7a38-54cd-43a3-a425-760846f4283a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:42.418979 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:42.419158 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 233/445] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Jul 13 23:01:42 2022] GET /identity/v3/groups/a4b41296b17f4bed8c47ea51f9a06baf => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:42.419926 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9e7a38-54cd-43a3-a425-760846f4283a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:42.500688 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:42.517220 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77cbdcf-d60f-4778-a5b8-16c5512f8fd9 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:42.526948 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9e7a38-54cd-43a3-a425-760846f4283a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:42.550384 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d25bdc-9828-43d4-86fd-4b52b65d2448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:42.551052 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d25bdc-9828-43d4-86fd-4b52b65d2448 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:42.551327 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d25bdc-9828-43d4-86fd-4b52b65d2448 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:42.551628 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d25bdc-9828-43d4-86fd-4b52b65d2448 None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:42.552384 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d25bdc-9828-43d4-86fd-4b52b65d2448 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:42.552686 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d25bdc-9828-43d4-86fd-4b52b65d2448 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:42.554071 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d25bdc-9828-43d4-86fd-4b52b65d2448 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:42.561217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9284982-b086-46ae-ab39-c7eff6c5b12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:42.561957 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9284982-b086-46ae-ab39-c7eff6c5b12c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:42.562240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9284982-b086-46ae-ab39-c7eff6c5b12c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:42.562516 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9284982-b086-46ae-ab39-c7eff6c5b12c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:42.562925 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 213/446] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:01:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:42.563763 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d25bdc-9828-43d4-86fd-4b52b65d2448 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:42.634176 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:42.636354 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9284982-b086-46ae-ab39-c7eff6c5b12c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:42.637860 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9284982-b086-46ae-ab39-c7eff6c5b12c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:42.665257 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d25bdc-9828-43d4-86fd-4b52b65d2448 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:42.699276 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1311a2-e8bc-47b5-a0d0-39f361d773a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:42.699966 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1311a2-e8bc-47b5-a0d0-39f361d773a4 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:42.700248 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1311a2-e8bc-47b5-a0d0-39f361d773a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:42.700524 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1311a2-e8bc-47b5-a0d0-39f361d773a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:42.701357 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1311a2-e8bc-47b5-a0d0-39f361d773a4 None admin] RBAC: Adding query filter params (group.id=a4b41296b17f4bed8c47ea51f9a06baf, role.id=5f0ca8371a394d339621748b570e78ee, scope.project.id=bd646bf2ad574160a5fa8742f70da34c) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:42.701663 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1311a2-e8bc-47b5-a0d0-39f361d773a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:42.703107 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1311a2-e8bc-47b5-a0d0-39f361d773a4 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:42.709655 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 214/447] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Jul 13 23:01:42 2022] GET /identity/v3/role_assignments?group.id=a4b41296b17f4bed8c47ea51f9a06baf&scope.project.id=bd646bf2ad574160a5fa8742f70da34c&role.id=5f0ca8371a394d339621748b570e78ee => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:42.710458 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1311a2-e8bc-47b5-a0d0-39f361d773a4 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:42.715798 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:42.744534 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 234/448] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:01:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 23:01:42.750212 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9284982-b086-46ae-ab39-c7eff6c5b12c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:42.848229 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1311a2-e8bc-47b5-a0d0-39f361d773a4 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:42.870002 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9284982-b086-46ae-ab39-c7eff6c5b12c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:42.895315 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067fbb9-ebc5-4e33-863e-732dae683f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:42.896151 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067fbb9-ebc5-4e33-863e-732dae683f95 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:42.896447 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067fbb9-ebc5-4e33-863e-732dae683f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:42.896724 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067fbb9-ebc5-4e33-863e-732dae683f95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:42.900892 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:42.917436 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf78522-ffa3-4615-8441-eaa9678a02a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:42.918244 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf78522-ffa3-4615-8441-eaa9678a02a8 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:42.918535 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf78522-ffa3-4615-8441-eaa9678a02a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:42.918816 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf78522-ffa3-4615-8441-eaa9678a02a8 None admin] PATH_INFO: `/v3/projects/c5ffbcb80ba6476698f912924febd384` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:42.921214 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf78522-ffa3-4615-8441-eaa9678a02a8 None admin] RBAC: Authorizing `identity:get_project(project_id=c5ffbcb80ba6476698f912924febd384)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:42.922705 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf78522-ffa3-4615-8441-eaa9678a02a8 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:42.924589 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 235/449] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:01:42 2022] GET /identity/v3/projects/c5ffbcb80ba6476698f912924febd384 => generated 344 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:01:42.930324 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf78522-ffa3-4615-8441-eaa9678a02a8 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:43.000667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b067fbb9-ebc5-4e33-863e-732dae683f95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:43.002448 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:43.003619 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b067fbb9-ebc5-4e33-863e-732dae683f95 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:43.075893 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf78522-ffa3-4615-8441-eaa9678a02a8 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:43.116082 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f2da4-94b0-4a92-984e-0c3a368038ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:43.117254 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f2da4-94b0-4a92-984e-0c3a368038ba None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:43.117533 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f2da4-94b0-4a92-984e-0c3a368038ba None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:43.117957 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f2da4-94b0-4a92-984e-0c3a368038ba None admin] PATH_INFO: `/v3/groups/149aa4b606fb4c1395ceb432e543c411` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:43.121244 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729f2da4-94b0-4a92-984e-0c3a368038ba None admin] RBAC: Authorizing `identity:get_group(group_id=149aa4b606fb4c1395ceb432e543c411)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:43.122716 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729f2da4-94b0-4a92-984e-0c3a368038ba None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:43.124786 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 236/450] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Jul 13 23:01:42 2022] GET /identity/v3/groups/149aa4b606fb4c1395ceb432e543c411 => generated 229 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:43.130968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f2da4-94b0-4a92-984e-0c3a368038ba None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:43.140178 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 215/451] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:01:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 23:01:43.146572 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067fbb9-ebc5-4e33-863e-732dae683f95 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:43.184746 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:43.251462 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f2da4-94b0-4a92-984e-0c3a368038ba None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:43.265442 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:43.267310 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067fbb9-ebc5-4e33-863e-732dae683f95 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:43.287724 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e282fe0-bef7-4778-bac8-7ce13fc19dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:43.288411 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e282fe0-bef7-4778-bac8-7ce13fc19dbf None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:43.288692 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e282fe0-bef7-4778-bac8-7ce13fc19dbf None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:43.288972 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e282fe0-bef7-4778-bac8-7ce13fc19dbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:43.290352 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e282fe0-bef7-4778-bac8-7ce13fc19dbf None admin] RBAC: Adding query filter params (group.id=149aa4b606fb4c1395ceb432e543c411, role.id=8cfd0b6e71e14e0983c3eb4d7478e027, scope.project.id=c5ffbcb80ba6476698f912924febd384) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:43.290666 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e282fe0-bef7-4778-bac8-7ce13fc19dbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:43.292133 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e282fe0-bef7-4778-bac8-7ce13fc19dbf None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:43.298006 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 237/452] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Jul 13 23:01:43 2022] GET /identity/v3/role_assignments?group.id=149aa4b606fb4c1395ceb432e543c411&scope.project.id=c5ffbcb80ba6476698f912924febd384&role.id=8cfd0b6e71e14e0983c3eb4d7478e027 => generated 611 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:01:43.301733 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff228a-9e60-4498-aab5-a6ee64a24ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:43.302614 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff228a-9e60-4498-aab5-a6ee64a24ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:43.302901 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff228a-9e60-4498-aab5-a6ee64a24ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:43.303176 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff228a-9e60-4498-aab5-a6ee64a24ec5 None admin] PATH_INFO: `/v3/projects/b64d10f5c7424d909a79d57916eaeac5` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:43.304498 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff228a-9e60-4498-aab5-a6ee64a24ec5 None admin] RBAC: Authorizing `identity:get_project(project_id=b64d10f5c7424d909a79d57916eaeac5)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:43.306039 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff228a-9e60-4498-aab5-a6ee64a24ec5 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:43.307921 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 216/453] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:01:43 2022] GET /identity/v3/projects/b64d10f5c7424d909a79d57916eaeac5 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:01:43.314030 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e282fe0-bef7-4778-bac8-7ce13fc19dbf None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:43.424005 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e282fe0-bef7-4778-bac8-7ce13fc19dbf None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:43.458918 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af13f526-eaf4-4b65-9b08-ac6a29ecfa70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:43.459730 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13f526-eaf4-4b65-9b08-ac6a29ecfa70 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:43.460018 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13f526-eaf4-4b65-9b08-ac6a29ecfa70 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:43.460293 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13f526-eaf4-4b65-9b08-ac6a29ecfa70 None admin] PATH_INFO: `/v3/groups/a4b41296b17f4bed8c47ea51f9a06baf` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:43.462000 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af13f526-eaf4-4b65-9b08-ac6a29ecfa70 None admin] RBAC: Authorizing `identity:get_group(group_id=a4b41296b17f4bed8c47ea51f9a06baf)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:43.463468 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af13f526-eaf4-4b65-9b08-ac6a29ecfa70 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:43.465180 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 238/454] 10.222.0.60 () {56 vars in 1180 bytes} [Wed Jul 13 23:01:43 2022] GET /identity/v3/groups/a4b41296b17f4bed8c47ea51f9a06baf => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:43.471053 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff228a-9e60-4498-aab5-a6ee64a24ec5 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:43.570753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff228a-9e60-4498-aab5-a6ee64a24ec5 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:43.573145 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:43.607922 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158531f-a8f2-40e0-9a53-8d09cd9a77ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:43.608576 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158531f-a8f2-40e0-9a53-8d09cd9a77ff None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:43.608857 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158531f-a8f2-40e0-9a53-8d09cd9a77ff None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:43.609129 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158531f-a8f2-40e0-9a53-8d09cd9a77ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:43.609974 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158531f-a8f2-40e0-9a53-8d09cd9a77ff None admin] RBAC: Adding query filter params (group.id=a4b41296b17f4bed8c47ea51f9a06baf, role.id=5f0ca8371a394d339621748b570e78ee, scope.project.id=b64d10f5c7424d909a79d57916eaeac5) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:43.610280 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158531f-a8f2-40e0-9a53-8d09cd9a77ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:43.611748 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8158531f-a8f2-40e0-9a53-8d09cd9a77ff None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:43.617814 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 217/455] 10.222.0.60 () {56 vars in 1376 bytes} [Wed Jul 13 23:01:43 2022] GET /identity/v3/role_assignments?group.id=a4b41296b17f4bed8c47ea51f9a06baf&scope.project.id=b64d10f5c7424d909a79d57916eaeac5&role.id=5f0ca8371a394d339621748b570e78ee => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:46.550255 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca0607-9a1b-4b1b-836b-03c2bf31a9bf None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:46.551294 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca0607-9a1b-4b1b-836b-03c2bf31a9bf None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:46.551810 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca0607-9a1b-4b1b-836b-03c2bf31a9bf None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:46.553754 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 239/456] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:46.559916 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c48bb0-fbb1-4525-89da-57f7416f8132 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:46.560886 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c48bb0-fbb1-4525-89da-57f7416f8132 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:46.561375 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c48bb0-fbb1-4525-89da-57f7416f8132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:46.584631 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-33c48bb0-fbb1-4525-89da-57f7416f8132 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:46.586192 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-33c48bb0-fbb1-4525-89da-57f7416f8132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:46.663304 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430307f5-340d-4d96-8f3f-5a74223284ac None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:46.663909 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430307f5-340d-4d96-8f3f-5a74223284ac None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:46.664190 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430307f5-340d-4d96-8f3f-5a74223284ac None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:46.668071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 240/457] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:46.675998 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277d57f-128a-4fa6-9a4f-3c852473681b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:46.676606 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277d57f-128a-4fa6-9a4f-3c852473681b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:46.677097 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277d57f-128a-4fa6-9a4f-3c852473681b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:46.700131 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 218/458] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:46.707760 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-7277d57f-128a-4fa6-9a4f-3c852473681b None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:46.709467 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7277d57f-128a-4fa6-9a4f-3c852473681b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:46.744116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c48bb0-fbb1-4525-89da-57f7416f8132 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:46.785909 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:46.824167 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 241/459] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 23:01:46.825598 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84310764-3311-4924-a9f9-5299017ab737 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:46.825891 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84310764-3311-4924-a9f9-5299017ab737 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:46.826163 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84310764-3311-4924-a9f9-5299017ab737 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:46.827731 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 242/460] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:46.835523 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5037e3-9de8-4efd-8b88-513b61e8b3dd None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:46.835896 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5037e3-9de8-4efd-8b88-513b61e8b3dd None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:46.836170 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5037e3-9de8-4efd-8b88-513b61e8b3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:46.875479 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-db5037e3-9de8-4efd-8b88-513b61e8b3dd None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:46.877509 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-db5037e3-9de8-4efd-8b88-513b61e8b3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:46.890407 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:46.898442 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c48bb0-fbb1-4525-89da-57f7416f8132 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:46.938562 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ceb59-4186-4979-874d-26f6bd9a0d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:46.939278 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ceb59-4186-4979-874d-26f6bd9a0d71 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:46.939618 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ceb59-4186-4979-874d-26f6bd9a0d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:46.939896 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ceb59-4186-4979-874d-26f6bd9a0d71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:46.952979 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693ceb59-4186-4979-874d-26f6bd9a0d71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:46.954507 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693ceb59-4186-4979-874d-26f6bd9a0d71 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:46.956940 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 243/461] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:46.957825 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5037e3-9de8-4efd-8b88-513b61e8b3dd None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:46.975846 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-693ceb59-4186-4979-874d-26f6bd9a0d71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:01:46.975846 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:46.975846 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:46.975846 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:46.975846 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:46.975846 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:46.975846 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:46.975846 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:46.975846 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:46.975846 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:46.975846 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:46.975846 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:46.975846 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:01:46.975846 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:01:46.975846 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:01:46.976763 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:01:46.976763 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:46.976763 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:46.976763 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:46.976763 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:46.976763 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:46.976763 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:46.976763 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:46.976763 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:46.976763 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:46.976763 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:46.976763 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:46.976763 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:46.976763 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:46.976763 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:46.976763 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:46.976763 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:46.978384 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:01:46.978384 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:01:46.978384 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:01:46.978384 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:46.978384 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 219/462] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:01:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:01:46.988732 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ceb59-4186-4979-874d-26f6bd9a0d71 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:47.061424 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5037e3-9de8-4efd-8b88-513b61e8b3dd None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:47.064020 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:47.099443 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d11fa78-dcf7-4dea-84a0-326cc7f72a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:47.100438 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11fa78-dcf7-4dea-84a0-326cc7f72a0d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:47.100721 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11fa78-dcf7-4dea-84a0-326cc7f72a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:47.101318 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d11fa78-dcf7-4dea-84a0-326cc7f72a0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:47.107680 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ceb59-4186-4979-874d-26f6bd9a0d71 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:47.114397 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d11fa78-dcf7-4dea-84a0-326cc7f72a0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:47.116338 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d11fa78-dcf7-4dea-84a0-326cc7f72a0d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:47.139480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-2d11fa78-dcf7-4dea-84a0-326cc7f72a0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:01:47.139480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:47.139480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:47.139480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:47.139480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:47.139480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:47.139480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:47.139480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:47.139480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:47.139480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:47.139480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:47.139480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:47.139480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:01:47.139480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:01:47.139480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:01:47.141523 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:01:47.141523 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:47.141523 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:47.141523 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:47.141523 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:47.141523 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:47.141523 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:47.141523 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:47.141523 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:47.141523 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:47.141523 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:47.141523 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:47.141523 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:47.141523 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:47.141523 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:47.141523 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:47.141523 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:47.143418 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:01:47.143418 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:01:47.143418 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:01:47.143418 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:47.145618 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29454459-48ee-4183-9a08-77050d70a383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:47.146112 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29454459-48ee-4183-9a08-77050d70a383 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:47.146268 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 244/463] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:01:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:47.146414 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29454459-48ee-4183-9a08-77050d70a383 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:47.146414 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29454459-48ee-4183-9a08-77050d70a383 None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:47.146715 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d11fa78-dcf7-4dea-84a0-326cc7f72a0d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:47.151556 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29454459-48ee-4183-9a08-77050d70a383 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:47.151556 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29454459-48ee-4183-9a08-77050d70a383 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:47.151556 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29454459-48ee-4183-9a08-77050d70a383 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:47.163453 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 220/464] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:01:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:47.164302 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29454459-48ee-4183-9a08-77050d70a383 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:47.177366 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:47.256247 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d11fa78-dcf7-4dea-84a0-326cc7f72a0d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:47.276206 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29454459-48ee-4183-9a08-77050d70a383 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:47.293883 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ef04ac-5b8a-4f98-a8db-ff207f191ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:47.294964 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef04ac-5b8a-4f98-a8db-ff207f191ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:47.295245 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef04ac-5b8a-4f98-a8db-ff207f191ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:47.295515 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ef04ac-5b8a-4f98-a8db-ff207f191ba1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:47.308913 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ef04ac-5b8a-4f98-a8db-ff207f191ba1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:47.310365 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ef04ac-5b8a-4f98-a8db-ff207f191ba1 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:47.310745 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e489f7e-96d2-4ea0-9635-e0249636c89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:47.311405 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e489f7e-96d2-4ea0-9635-e0249636c89f None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:47.311722 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e489f7e-96d2-4ea0-9635-e0249636c89f None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:47.311998 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e489f7e-96d2-4ea0-9635-e0249636c89f None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:47.312725 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e489f7e-96d2-4ea0-9635-e0249636c89f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:47.313034 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e489f7e-96d2-4ea0-9635-e0249636c89f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:47.314400 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e489f7e-96d2-4ea0-9635-e0249636c89f None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:47.331904 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 221/465] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:01:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:47.331904 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7e11c-7fb8-46b5-8dc1-ccc3376183c7 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:47.331904 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7e11c-7fb8-46b5-8dc1-ccc3376183c7 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:47.331904 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7e11c-7fb8-46b5-8dc1-ccc3376183c7 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:47.331904 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 222/466] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:47.331904 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f7e11c-7fb8-46b5-8dc1-ccc3376183c7 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:47.335728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-49ef04ac-5b8a-4f98-a8db-ff207f191ba1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:01:47.335728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:47.335728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:47.335728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:47.335728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:47.335728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:47.335728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:47.335728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:47.335728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:47.335728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:47.335728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:47.335728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:47.335728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:01:47.335728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:01:47.335728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:01:47.337146 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:01:47.337146 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:47.337146 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:47.337146 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:47.337146 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:47.337146 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:47.337146 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:47.337146 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:47.337146 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:47.337146 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:47.337146 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:47.337146 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:47.337146 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:47.337146 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:47.337146 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:47.337146 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:47.337146 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:47.337970 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:01:47.337970 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:01:47.337970 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:01:47.337970 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:47.338627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 245/467] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:01:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:47.339204 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ef04ac-5b8a-4f98-a8db-ff207f191ba1 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:47.351405 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:47.414332 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:47.452080 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ef04ac-5b8a-4f98-a8db-ff207f191ba1 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:47.453584 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f7e11c-7fb8-46b5-8dc1-ccc3376183c7 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:47.487944 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df253b-a3c5-4b6e-b32c-83f019837737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:47.488613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df253b-a3c5-4b6e-b32c-83f019837737 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:47.488888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df253b-a3c5-4b6e-b32c-83f019837737 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:47.489159 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df253b-a3c5-4b6e-b32c-83f019837737 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:47.489852 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1df253b-a3c5-4b6e-b32c-83f019837737 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:47.490025 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dab517-fba9-43ad-a409-c50b86b921af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:47.490650 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dab517-fba9-43ad-a409-c50b86b921af None admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:47.490930 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dab517-fba9-43ad-a409-c50b86b921af None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:47.491203 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dab517-fba9-43ad-a409-c50b86b921af None admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:47.491297 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1df253b-a3c5-4b6e-b32c-83f019837737 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:47.491939 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dab517-fba9-43ad-a409-c50b86b921af None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:47.493413 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dab517-fba9-43ad-a409-c50b86b921af None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:47.519651 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 246/468] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Jul 13 23:01:47 2022] POST /identity/v3/users => generated 248 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:01:47.519651 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08734b75-64e9-4232-bcf1-0a894bc13812 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:47.519651 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08734b75-64e9-4232-bcf1-0a894bc13812 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:47.519651 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08734b75-64e9-4232-bcf1-0a894bc13812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:47.522212 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 223/469] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Jul 13 23:01:47 2022] POST /identity/v3/users => generated 246 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:01:47.531752 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dab517-fba9-43ad-a409-c50b86b921af None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:47.547899 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-08734b75-64e9-4232-bcf1-0a894bc13812 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:47.549223 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-08734b75-64e9-4232-bcf1-0a894bc13812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:47.675842 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:47.698035 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 247/470] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:47.722732 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dab517-fba9-43ad-a409-c50b86b921af None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:47.729291 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08734b75-64e9-4232-bcf1-0a894bc13812 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:47.749175 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:47.762830 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262720d5-09a8-41d2-b1e6-720aec8e2c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:47.764097 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262720d5-09a8-41d2-b1e6-720aec8e2c62 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:47.764376 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262720d5-09a8-41d2-b1e6-720aec8e2c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:47.764648 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262720d5-09a8-41d2-b1e6-720aec8e2c62 None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:47.765450 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262720d5-09a8-41d2-b1e6-720aec8e2c62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:47.765754 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262720d5-09a8-41d2-b1e6-720aec8e2c62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:47.767202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262720d5-09a8-41d2-b1e6-720aec8e2c62 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:47.780551 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 224/471] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:01:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:47.786592 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262720d5-09a8-41d2-b1e6-720aec8e2c62 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:47.863327 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08734b75-64e9-4232-bcf1-0a894bc13812 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:47.910047 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262720d5-09a8-41d2-b1e6-720aec8e2c62 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:47.911359 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f501a63-2ac0-4588-8b33-d5eab225c30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:47.912463 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f501a63-2ac0-4588-8b33-d5eab225c30d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:47.912956 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f501a63-2ac0-4588-8b33-d5eab225c30d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:47.913444 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f501a63-2ac0-4588-8b33-d5eab225c30d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:47.927202 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f501a63-2ac0-4588-8b33-d5eab225c30d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:47.928832 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f501a63-2ac0-4588-8b33-d5eab225c30d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:47.949221 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-1f501a63-2ac0-4588-8b33-d5eab225c30d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:01:47.949221 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:47.949221 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:47.949221 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:47.949221 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:47.949221 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:47.949221 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:47.949221 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:47.949221 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:47.949221 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:47.949221 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:47.949221 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:47.949221 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:01:47.949221 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:01:47.949221 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:01:47.950114 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:01:47.950114 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:47.950114 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:47.950114 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:47.950114 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:47.950114 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:47.950114 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:47.950114 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:47.950114 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:47.950114 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:47.950114 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:47.950114 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:47.950114 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:47.950114 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:47.950114 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:47.950114 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:47.950114 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:47.950938 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:01:47.950938 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:01:47.950938 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:01:47.950938 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:47.953099 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ec9ee-8407-4b94-a42e-b492ab2af36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:47.953774 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ec9ee-8407-4b94-a42e-b492ab2af36e None admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:47.954057 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ec9ee-8407-4b94-a42e-b492ab2af36e None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:47.954329 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ec9ee-8407-4b94-a42e-b492ab2af36e None admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:47.955041 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ec9ee-8407-4b94-a42e-b492ab2af36e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:47.955145 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 248/472] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:01:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:01:47.958169 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ec9ee-8407-4b94-a42e-b492ab2af36e None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:47.958307 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f014043-98ad-4b92-9d28-ba1579a337f9 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:47.958307 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f014043-98ad-4b92-9d28-ba1579a337f9 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:47.958307 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f014043-98ad-4b92-9d28-ba1579a337f9 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:47.958767 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 249/473] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:47.962539 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f014043-98ad-4b92-9d28-ba1579a337f9 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:47.986567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 225/474] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Jul 13 23:01:47 2022] POST /identity/v3/users => generated 248 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:01:47.989202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa83e28-dc29-4e74-8bb9-b0f6f7a5e161 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:47.989976 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa83e28-dc29-4e74-8bb9-b0f6f7a5e161 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:47.990510 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa83e28-dc29-4e74-8bb9-b0f6f7a5e161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:48.015116 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:48.015581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-5fa83e28-dc29-4e74-8bb9-b0f6f7a5e161 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:48.018775 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-5fa83e28-dc29-4e74-8bb9-b0f6f7a5e161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:48.081817 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:48.092119 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f014043-98ad-4b92-9d28-ba1579a337f9 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:48.134112 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 226/475] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:48.139594 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695ba5c-7ce5-41f4-b665-7f5b1d1492cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:48.139594 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695ba5c-7ce5-41f4-b665-7f5b1d1492cb None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:48.139594 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695ba5c-7ce5-41f4-b665-7f5b1d1492cb None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:48.139594 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695ba5c-7ce5-41f4-b665-7f5b1d1492cb None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:48.139955 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8695ba5c-7ce5-41f4-b665-7f5b1d1492cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:48.139955 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8695ba5c-7ce5-41f4-b665-7f5b1d1492cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:48.139955 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8695ba5c-7ce5-41f4-b665-7f5b1d1492cb None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:48.155669 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 250/476] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:01:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:01:48.158055 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa83e28-dc29-4e74-8bb9-b0f6f7a5e161 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:48.175329 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695ba5c-7ce5-41f4-b665-7f5b1d1492cb None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:48.300200 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa83e28-dc29-4e74-8bb9-b0f6f7a5e161 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:48.336235 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695ba5c-7ce5-41f4-b665-7f5b1d1492cb None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:48.338753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b30e5a-3801-4050-9848-f939b8dc77bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:48.339440 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b30e5a-3801-4050-9848-f939b8dc77bc None admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:48.339748 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b30e5a-3801-4050-9848-f939b8dc77bc None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:48.340026 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b30e5a-3801-4050-9848-f939b8dc77bc None admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:48.340747 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b30e5a-3801-4050-9848-f939b8dc77bc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:48.342208 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b30e5a-3801-4050-9848-f939b8dc77bc None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:48.343500 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:48.379362 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e613e9ee-9cfa-431c-b584-6d43c10821db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:48.379741 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 227/477] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Jul 13 23:01:48 2022] POST /identity/v3/users => generated 249 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:01:48.380217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e613e9ee-9cfa-431c-b584-6d43c10821db None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:48.380498 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e613e9ee-9cfa-431c-b584-6d43c10821db None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:48.380770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e613e9ee-9cfa-431c-b584-6d43c10821db None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:48.381602 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e613e9ee-9cfa-431c-b584-6d43c10821db None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:48.383043 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e613e9ee-9cfa-431c-b584-6d43c10821db None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:48.391665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.sql.core [None req-e613e9ee-9cfa-431c-b584-6d43c10821db None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 13 23:01:48.391665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 13 23:01:48.391665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [parameters: {'id': '514c11f0e9b2469f85e3a3d9b79b0110', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 13 23:01:48.391665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80284) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 13 23:01:48.411623 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-e613e9ee-9cfa-431c-b584-6d43c10821db None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 13 23:01:48.411623 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:48.411623 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 13 23:01:48.411623 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 13 23:01:48.411623 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 13 23:01:48.411623 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 13 23:01:48.411623 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 13 23:01:48.411623 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application result = self._query(query) Jul 13 23:01:48.411623 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 13 23:01:48.411623 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application conn.query(q) Jul 13 23:01:48.411623 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 13 23:01:48.411623 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 13 23:01:48.411623 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 13 23:01:48.411623 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application result.read() Jul 13 23:01:48.411623 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 13 23:01:48.412536 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 13 23:01:48.412536 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 13 23:01:48.412536 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 13 23:01:48.412536 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 13 23:01:48.412536 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 13 23:01:48.412536 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 13 23:01:48.412536 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 13 23:01:48.412536 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 13 23:01:48.412536 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:48.412536 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 13 23:01:48.412536 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:48.412536 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:48.412536 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 13 23:01:48.412536 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 13 23:01:48.412536 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 13 23:01:48.412536 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return ref.to_dict() Jul 13 23:01:48.412536 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 13 23:01:48.412536 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application next(self.gen) Jul 13 23:01:48.413520 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 13 23:01:48.413520 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application yield resource Jul 13 23:01:48.413520 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 13 23:01:48.413520 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application next(self.gen) Jul 13 23:01:48.413520 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 13 23:01:48.413520 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self.session.rollback() Jul 13 23:01:48.413520 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 13 23:01:48.413520 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self.force_reraise() Jul 13 23:01:48.413520 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 13 23:01:48.413520 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise self.value Jul 13 23:01:48.413520 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 13 23:01:48.413520 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 13 23:01:48.413520 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 13 23:01:48.413520 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application session.commit() Jul 13 23:01:48.413520 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 13 23:01:48.413520 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 13 23:01:48.413520 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 13 23:01:48.414452 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._prepare_impl() Jul 13 23:01:48.414452 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 13 23:01:48.414452 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self.session.flush() Jul 13 23:01:48.414452 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 13 23:01:48.414452 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._flush(objects) Jul 13 23:01:48.414452 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 13 23:01:48.414452 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 13 23:01:48.414452 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 13 23:01:48.414452 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application compat.raise_( Jul 13 23:01:48.414452 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 13 23:01:48.414452 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception Jul 13 23:01:48.414452 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 13 23:01:48.414452 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application flush_context.execute() Jul 13 23:01:48.414452 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 13 23:01:48.414452 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rec.execute(self) Jul 13 23:01:48.414452 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 13 23:01:48.414452 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 13 23:01:48.415872 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 13 23:01:48.415872 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 13 23:01:48.415872 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 13 23:01:48.415872 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 13 23:01:48.415872 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 13 23:01:48.415872 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 13 23:01:48.415872 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 13 23:01:48.415872 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 13 23:01:48.415872 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 13 23:01:48.415872 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 13 23:01:48.415872 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 13 23:01:48.415872 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 13 23:01:48.415872 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 13 23:01:48.415872 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 13 23:01:48.415872 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 13 23:01:48.415872 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception Jul 13 23:01:48.416683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 13 23:01:48.416683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 13 23:01:48.416683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 13 23:01:48.416683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 13 23:01:48.416683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 13 23:01:48.416683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application result = self._query(query) Jul 13 23:01:48.416683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 13 23:01:48.416683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application conn.query(q) Jul 13 23:01:48.416683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 13 23:01:48.416683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 13 23:01:48.416683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 13 23:01:48.416683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application result.read() Jul 13 23:01:48.416683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 13 23:01:48.416683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 13 23:01:48.416683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 13 23:01:48.416683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 13 23:01:48.417461 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 13 23:01:48.417461 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 13 23:01:48.417461 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 13 23:01:48.417461 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 13 23:01:48.417461 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 13 23:01:48.417461 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 13 23:01:48.417461 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [parameters: {'id': '514c11f0e9b2469f85e3a3d9b79b0110', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 13 23:01:48.417461 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 13 23:01:48.417461 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:48.417461 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 23:01:48.417461 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:48.417461 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:48.417461 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:48.417461 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:48.417461 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:48.417461 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:48.418263 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:48.418263 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:48.418263 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:48.418263 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:48.418263 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:48.418263 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:48.418263 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 13 23:01:48.418263 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 13 23:01:48.418263 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:48.418263 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:48.418263 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 13 23:01:48.418263 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 13 23:01:48.418263 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 13 23:01:48.418263 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 13 23:01:48.418263 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 13 23:01:48.418263 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:48.419097 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 251/478] 10.222.0.60 () {60 vars in 1131 bytes} [Wed Jul 13 23:01:48 2022] POST /identity/v3/roles => generated 144 bytes in 238 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:48.420173 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b30e5a-3801-4050-9848-f939b8dc77bc None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:48.465326 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:48.557053 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b30e5a-3801-4050-9848-f939b8dc77bc None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:48.598280 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f7ad9-4ba5-4035-b01d-b3d644cdef7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:48.599021 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f7ad9-4ba5-4035-b01d-b3d644cdef7c None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:48.599357 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f7ad9-4ba5-4035-b01d-b3d644cdef7c None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:48.599662 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f7ad9-4ba5-4035-b01d-b3d644cdef7c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:48.614884 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491f7ad9-4ba5-4035-b01d-b3d644cdef7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:48.624106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-491f7ad9-4ba5-4035-b01d-b3d644cdef7c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 23:01:48.624106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:48.624106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:48.624106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:48.624106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:48.624106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:48.624106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:48.624106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:48.624106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:48.624106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:48.624106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:48.624106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:48.624106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:48.624106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:48.624106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:48.624106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:48.625001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:48.625001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:48.625001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:48.625001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:48.625001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:48.625001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:48.625001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:48.625001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:48.625001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:48.625001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:48.625001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:48.625001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:48.625001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:48.625001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:48.625001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:48.625001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:48.625784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:48.625784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:48.625784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:48.625784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:48.625784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:48.625784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:48.625784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:48.625784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:48.625784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:48.625784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:48.625784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 23:01:48.625784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:48.625784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 228/479] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jul 13 23:01:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:48.631355 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e613e9ee-9cfa-431c-b584-6d43c10821db None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:48.752638 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e613e9ee-9cfa-431c-b584-6d43c10821db None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:48.791296 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c6d794-4abf-4f3b-b2c9-700045beddbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:48.792341 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6d794-4abf-4f3b-b2c9-700045beddbb None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:48.792879 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6d794-4abf-4f3b-b2c9-700045beddbb None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:48.793345 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6d794-4abf-4f3b-b2c9-700045beddbb None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:48.794311 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c6d794-4abf-4f3b-b2c9-700045beddbb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:48.794813 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c6d794-4abf-4f3b-b2c9-700045beddbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:48.796411 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c6d794-4abf-4f3b-b2c9-700045beddbb None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:48.805469 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 252/480] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:01:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:49.996603 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d565e-9372-4818-b063-edd880fb279e None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:49.997152 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d565e-9372-4818-b063-edd880fb279e None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:49.997152 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d565e-9372-4818-b063-edd880fb279e None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:49.998759 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 229/481] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:50.004198 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342b705-a1ef-407d-9125-46f15003812e None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:50.004624 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342b705-a1ef-407d-9125-46f15003812e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:50.004818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342b705-a1ef-407d-9125-46f15003812e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:50.028523 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-7342b705-a1ef-407d-9125-46f15003812e None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:50.029722 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7342b705-a1ef-407d-9125-46f15003812e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:50.059074 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:50.121192 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 253/482] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:50.163222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392d565e-9372-4818-b063-edd880fb279e None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:50.197824 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d5bf42-a362-4001-be00-abced9a3c714 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:50.198129 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d5bf42-a362-4001-be00-abced9a3c714 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:50.198362 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d5bf42-a362-4001-be00-abced9a3c714 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:50.200022 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 254/483] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:01:50.205229 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b2972-591b-461b-85c8-bd550763eee4 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:50.205792 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b2972-591b-461b-85c8-bd550763eee4 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:50.206070 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b2972-591b-461b-85c8-bd550763eee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:50.225678 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-0c7b2972-591b-461b-85c8-bd550763eee4 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:50.227809 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0c7b2972-591b-461b-85c8-bd550763eee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:50.306808 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:50.324682 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392d565e-9372-4818-b063-edd880fb279e None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:50.368686 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce87d4e4-0b79-4738-86cc-ce4f695ebcd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:50.369487 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87d4e4-0b79-4738-86cc-ce4f695ebcd7 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:50.369766 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87d4e4-0b79-4738-86cc-ce4f695ebcd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:50.370037 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87d4e4-0b79-4738-86cc-ce4f695ebcd7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:50.377389 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 255/484] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 23:01:50.386051 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce87d4e4-0b79-4738-86cc-ce4f695ebcd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:50.399941 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-ce87d4e4-0b79-4738-86cc-ce4f695ebcd7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:01:50.399941 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:50.399941 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:50.399941 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:50.399941 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:50.399941 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:50.399941 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:50.399941 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:50.399941 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:50.399941 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:50.399941 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:50.399941 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:50.399941 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:50.399941 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:50.399941 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:50.399941 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:50.402124 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:50.402124 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:50.402124 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:50.402124 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:50.402124 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:50.402124 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:50.402124 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:50.402124 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:50.402124 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:50.402124 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:50.402124 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:50.402124 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:50.402124 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:50.402124 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:50.402124 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:50.402124 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:50.404183 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:50.404183 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:50.404183 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:50.404183 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:50.404183 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:50.404183 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:50.404183 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:50.404183 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:50.404183 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:50.404183 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:50.404183 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:01:50.404183 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:50.404183 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 230/485] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jul 13 23:01:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:50.416669 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b2972-591b-461b-85c8-bd550763eee4 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:50.419468 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce87d4e4-0b79-4738-86cc-ce4f695ebcd7 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:50.526455 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce87d4e4-0b79-4738-86cc-ce4f695ebcd7 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:50.563327 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b1b9f-77a2-42aa-a7c1-04a65d18fe19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:50.564077 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b1b9f-77a2-42aa-a7c1-04a65d18fe19 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:50.564356 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b1b9f-77a2-42aa-a7c1-04a65d18fe19 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:50.564627 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b1b9f-77a2-42aa-a7c1-04a65d18fe19 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:50.576888 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85b1b9f-77a2-42aa-a7c1-04a65d18fe19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:50.584926 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-b85b1b9f-77a2-42aa-a7c1-04a65d18fe19 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:01:50.584926 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:50.584926 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:50.584926 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:50.584926 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:50.584926 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:50.584926 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:50.584926 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:50.584926 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:50.584926 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:50.584926 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:50.584926 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:50.584926 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:50.584926 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:50.584926 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:50.584926 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:50.587174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:50.587174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:50.587174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:50.587174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:50.587174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:50.587174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:50.587174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:50.587174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:50.587174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:50.587174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:50.587174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:50.587174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:50.587174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:50.587174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:50.587174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:50.587174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:50.589222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:50.589222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:50.589222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:50.589222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:50.589222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:50.589222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:50.589222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:50.589222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:50.589222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:50.589222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:50.589222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:01:50.589222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:50.589222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 231/486] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jul 13 23:01:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:01:50.595660 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b1b9f-77a2-42aa-a7c1-04a65d18fe19 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:50.613323 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:50.633112 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:50.650327 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b2972-591b-461b-85c8-bd550763eee4 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:50.719197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c657860b-56d9-4592-9220-b2da6042a286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:50.719899 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657860b-56d9-4592-9220-b2da6042a286 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:50.720181 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657860b-56d9-4592-9220-b2da6042a286 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:50.720452 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657860b-56d9-4592-9220-b2da6042a286 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:50.724122 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c657860b-56d9-4592-9220-b2da6042a286 None admin] RBAC: Adding query filter params (name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:50.724438 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c657860b-56d9-4592-9220-b2da6042a286 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:50.725837 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c657860b-56d9-4592-9220-b2da6042a286 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:50.746953 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 256/487] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jul 13 23:01:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:50.764580 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c657860b-56d9-4592-9220-b2da6042a286 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:50.772370 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b1b9f-77a2-42aa-a7c1-04a65d18fe19 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:50.809121 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f186a-8cf6-4cd0-a2ea-73ba8568f665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:50.809783 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f186a-8cf6-4cd0-a2ea-73ba8568f665 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:50.810063 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f186a-8cf6-4cd0-a2ea-73ba8568f665 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:50.810332 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f186a-8cf6-4cd0-a2ea-73ba8568f665 None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:50.811233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f186a-8cf6-4cd0-a2ea-73ba8568f665 None admin] RBAC: Adding query filter params (name=service) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:50.811583 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f186a-8cf6-4cd0-a2ea-73ba8568f665 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:50.815628 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f186a-8cf6-4cd0-a2ea-73ba8568f665 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:50.827990 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 232/488] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jul 13 23:01:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:50.829328 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f186a-8cf6-4cd0-a2ea-73ba8568f665 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:50.903370 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:50.946801 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f186a-8cf6-4cd0-a2ea-73ba8568f665 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:50.969771 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c657860b-56d9-4592-9220-b2da6042a286 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:50.988315 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59d7e1-4b6c-406f-a3cc-055dad0e90f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:50.989542 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59d7e1-4b6c-406f-a3cc-055dad0e90f2 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:50.989827 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59d7e1-4b6c-406f-a3cc-055dad0e90f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:50.990100 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59d7e1-4b6c-406f-a3cc-055dad0e90f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:51.001882 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb59d7e1-4b6c-406f-a3cc-055dad0e90f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:51.003343 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb59d7e1-4b6c-406f-a3cc-055dad0e90f2 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:51.008105 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb79e431-06f5-4069-bacf-7d8cac77c843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:51.008900 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79e431-06f5-4069-bacf-7d8cac77c843 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:51.009183 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79e431-06f5-4069-bacf-7d8cac77c843 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:51.009456 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb79e431-06f5-4069-bacf-7d8cac77c843 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:51.020525 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb79e431-06f5-4069-bacf-7d8cac77c843 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:51.020772 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-bb59d7e1-4b6c-406f-a3cc-055dad0e90f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:01:51.020772 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:51.020772 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:51.020772 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:51.020772 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:51.020772 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:51.020772 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:51.020772 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:51.020772 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:51.020772 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:51.020772 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:51.020772 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:51.020772 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:01:51.020772 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:01:51.020772 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:01:51.021567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:01:51.021567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:51.021567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:51.021567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:51.021567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:51.021567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:51.021567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:51.021567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:51.021567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:51.021567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:51.021567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:51.021567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:51.021567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:51.021567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:51.021567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:51.021567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:51.021567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:51.027290 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:01:51.027290 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:01:51.027290 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:01:51.027290 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:51.027499 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb79e431-06f5-4069-bacf-7d8cac77c843 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:51.027594 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 233/489] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:01:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:51.027594 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e6bdd-9a29-44a5-bed9-46b56658cc63 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:51.027594 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e6bdd-9a29-44a5-bed9-46b56658cc63 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:51.027594 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e6bdd-9a29-44a5-bed9-46b56658cc63 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:51.027594 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 234/490] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:51.027594 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1b6bf-2c1f-4f05-abf8-53b54745dc2d None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:51.027594 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1b6bf-2c1f-4f05-abf8-53b54745dc2d None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:51.028005 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1b6bf-2c1f-4f05-abf8-53b54745dc2d None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:51.028578 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 235/491] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:51.029128 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce1b6bf-2c1f-4f05-abf8-53b54745dc2d None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:51.051615 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-eb79e431-06f5-4069-bacf-7d8cac77c843 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:01:51.051615 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:51.051615 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:51.051615 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:51.051615 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:51.051615 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:51.051615 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:51.051615 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:51.051615 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:51.051615 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:51.051615 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:51.051615 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:51.051615 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:01:51.051615 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:01:51.051615 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:01:51.052537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:01:51.052537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:51.052537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:51.052537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:51.052537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:51.052537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:51.052537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:51.052537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:51.052537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:51.052537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:51.052537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:51.052537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:51.052537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:51.052537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:51.052537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:51.052537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:51.052537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:51.059325 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:01:51.059325 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:01:51.059325 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:01:51.059325 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:51.059325 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 257/492] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:01:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:51.059325 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd6987-ab14-4ac7-b194-679541500d79 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:51.059325 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd6987-ab14-4ac7-b194-679541500d79 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:51.059325 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd6987-ab14-4ac7-b194-679541500d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:51.069616 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:51.076726 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-95fd6987-ab14-4ac7-b194-679541500d79 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:51.077950 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-95fd6987-ab14-4ac7-b194-679541500d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:51.145558 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce1b6bf-2c1f-4f05-abf8-53b54745dc2d None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:51.163555 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 258/493] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:51.163555 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dca778-5d86-4f74-9c1f-98dc27924ca1 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:51.163555 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dca778-5d86-4f74-9c1f-98dc27924ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:51.163555 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dca778-5d86-4f74-9c1f-98dc27924ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:51.168253 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:51.181761 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-77dca778-5d86-4f74-9c1f-98dc27924ca1 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:51.182718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eafc0ab-64cb-4780-9908-20eed21c0580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:51.183055 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-77dca778-5d86-4f74-9c1f-98dc27924ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:51.183506 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafc0ab-64cb-4780-9908-20eed21c0580 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:51.183877 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafc0ab-64cb-4780-9908-20eed21c0580 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:51.184221 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eafc0ab-64cb-4780-9908-20eed21c0580 None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:51.185022 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eafc0ab-64cb-4780-9908-20eed21c0580 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:51.185483 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eafc0ab-64cb-4780-9908-20eed21c0580 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:51.186710 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eafc0ab-64cb-4780-9908-20eed21c0580 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:51.199707 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 236/494] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:01:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:51.204391 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eafc0ab-64cb-4780-9908-20eed21c0580 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:51.263581 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 259/495] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:51.263581 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dca778-5d86-4f74-9c1f-98dc27924ca1 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:51.324831 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eafc0ab-64cb-4780-9908-20eed21c0580 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:51.338343 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:51.370437 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624c264-e3a9-48b6-be50-04690575a3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:51.371088 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624c264-e3a9-48b6-be50-04690575a3c4 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:51.371428 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624c264-e3a9-48b6-be50-04690575a3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:51.371752 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624c264-e3a9-48b6-be50-04690575a3c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:51.372511 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4624c264-e3a9-48b6-be50-04690575a3c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:51.372822 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4624c264-e3a9-48b6-be50-04690575a3c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:51.374293 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4624c264-e3a9-48b6-be50-04690575a3c4 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:51.383996 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dca778-5d86-4f74-9c1f-98dc27924ca1 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:51.392583 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 237/496] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:01:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:51.392583 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624c264-e3a9-48b6-be50-04690575a3c4 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:51.426976 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f05a354-7d13-493f-ba2d-7f12c591aad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:51.427942 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05a354-7d13-493f-ba2d-7f12c591aad2 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:51.428289 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05a354-7d13-493f-ba2d-7f12c591aad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:51.428625 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05a354-7d13-493f-ba2d-7f12c591aad2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:51.438498 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:51.441317 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f05a354-7d13-493f-ba2d-7f12c591aad2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:51.450449 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-2f05a354-7d13-493f-ba2d-7f12c591aad2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:01:51.450449 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:51.450449 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:51.450449 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:51.450449 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:51.450449 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:51.450449 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:51.450449 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:51.450449 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:51.450449 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:51.450449 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:51.450449 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:51.450449 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:51.450449 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:51.450449 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:51.450449 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:51.451468 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:51.451468 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:51.451468 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:51.451468 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:51.451468 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:51.451468 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:51.451468 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:51.451468 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:51.451468 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:51.451468 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:51.451468 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:51.451468 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:51.451468 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:51.451468 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:51.451468 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:51.451468 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:51.452292 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:51.452292 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:51.452292 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:51.452292 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:51.452292 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:51.452292 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:51.452292 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:51.452292 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:51.452292 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:51.452292 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:51.452292 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:01:51.452292 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:51.452292 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 260/497] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jul 13 23:01:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:51.452928 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f05a354-7d13-493f-ba2d-7f12c591aad2 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:51.513483 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624c264-e3a9-48b6-be50-04690575a3c4 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:51.553192 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da9464-ac38-4791-8b33-53bcdd28ca12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:51.553993 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da9464-ac38-4791-8b33-53bcdd28ca12 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:51.554272 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da9464-ac38-4791-8b33-53bcdd28ca12 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:51.554551 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da9464-ac38-4791-8b33-53bcdd28ca12 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:51.563438 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60da9464-ac38-4791-8b33-53bcdd28ca12 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:51.565001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60da9464-ac38-4791-8b33-53bcdd28ca12 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:51.572407 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f05a354-7d13-493f-ba2d-7f12c591aad2 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:51.582021 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-60da9464-ac38-4791-8b33-53bcdd28ca12 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 23:01:51.582021 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:51.582021 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:51.582021 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:51.582021 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:51.582021 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:51.582021 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:51.582021 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:51.582021 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:51.582021 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:51.582021 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:51.582021 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:51.582021 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 23:01:51.582021 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 23:01:51.582021 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 23:01:51.582021 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 23:01:51.583140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:51.583140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:51.583140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 23:01:51.583140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:01:51.583140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 23:01:51.583140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:01:51.583140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:51.583140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:51.583140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:51.583140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:51.583140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:51.583140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:51.583140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:51.583140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:51.583140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:51.583140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:51.583140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:51.583140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:51.584362 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:51.584362 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:51.584362 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 23:01:51.584362 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 23:01:51.584362 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 23:01:51.584362 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 23:01:51.584362 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 23:01:51.584362 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 23:01:51.584362 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 23:01:51.584362 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:51.587701 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 238/498] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Jul 13 23:01:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:51.588854 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da9464-ac38-4791-8b33-53bcdd28ca12 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:51.599100 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:51.608579 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad66a67-ec55-4021-8f2f-28d683a43a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:51.609404 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad66a67-ec55-4021-8f2f-28d683a43a74 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:51.609741 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad66a67-ec55-4021-8f2f-28d683a43a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:51.610018 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad66a67-ec55-4021-8f2f-28d683a43a74 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:51.621482 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad66a67-ec55-4021-8f2f-28d683a43a74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:51.629302 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-8ad66a67-ec55-4021-8f2f-28d683a43a74 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:01:51.629302 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:51.629302 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:51.629302 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:51.629302 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:51.629302 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:51.629302 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:51.629302 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:51.629302 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:51.629302 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:51.629302 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:51.629302 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:51.629302 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:51.629302 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:51.629302 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:51.629302 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:51.630665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:51.630665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:51.630665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:51.630665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:51.630665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:51.630665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:51.630665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:51.630665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:51.630665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:51.630665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:51.630665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:51.630665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:51.630665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:51.630665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:51.630665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:51.630665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:51.632139 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:51.632139 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:51.632139 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:51.632139 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:51.632139 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:51.632139 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:51.632139 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:51.632139 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:51.632139 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:51.632139 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:51.632139 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:01:51.632139 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:51.632139 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 261/499] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jul 13 23:01:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:51.632139 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad66a67-ec55-4021-8f2f-28d683a43a74 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:51.694274 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:51.698008 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da9464-ac38-4791-8b33-53bcdd28ca12 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:51.734403 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd33f08-98a5-4184-b5fc-9e3da2955f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:51.735199 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd33f08-98a5-4184-b5fc-9e3da2955f13 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:51.735482 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd33f08-98a5-4184-b5fc-9e3da2955f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:51.735737 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad66a67-ec55-4021-8f2f-28d683a43a74 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:51.735850 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd33f08-98a5-4184-b5fc-9e3da2955f13 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:51.744468 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd33f08-98a5-4184-b5fc-9e3da2955f13 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:51.745950 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd33f08-98a5-4184-b5fc-9e3da2955f13 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:51.760045 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-6cd33f08-98a5-4184-b5fc-9e3da2955f13 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 23:01:51.760045 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:51.760045 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:51.760045 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:51.760045 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:51.760045 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:51.760045 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:51.760045 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:51.760045 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:51.760045 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:51.760045 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:51.760045 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:51.760045 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 23:01:51.760045 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 23:01:51.760045 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 23:01:51.760045 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 23:01:51.761230 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:51.761230 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:51.761230 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 23:01:51.761230 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:01:51.761230 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 23:01:51.761230 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:01:51.761230 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:51.761230 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:51.761230 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:51.761230 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:51.761230 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:51.761230 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:51.761230 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:51.761230 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:51.761230 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:51.761230 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:51.761230 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:51.761230 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:51.762183 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:51.762183 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:51.762183 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 23:01:51.762183 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 23:01:51.762183 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 23:01:51.762183 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 23:01:51.762183 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 23:01:51.762183 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 23:01:51.762183 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 23:01:51.762183 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:51.762183 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 239/500] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:01:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:51.762760 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e5133-0b2b-4cb0-8ca0-9c9b228b9370 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:51.762760 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e5133-0b2b-4cb0-8ca0-9c9b228b9370 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:51.763214 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e5133-0b2b-4cb0-8ca0-9c9b228b9370 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:51.767645 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 240/501] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:51.767645 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561e5133-0b2b-4cb0-8ca0-9c9b228b9370 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:51.782056 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f81f5-7bba-4f31-96d5-e0b0b994a3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:51.783189 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f81f5-7bba-4f31-96d5-e0b0b994a3bb None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:51.783189 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f81f5-7bba-4f31-96d5-e0b0b994a3bb None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:51.783415 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f81f5-7bba-4f31-96d5-e0b0b994a3bb None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:51.784089 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458f81f5-7bba-4f31-96d5-e0b0b994a3bb None admin] RBAC: Adding query filter params (name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:51.784395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458f81f5-7bba-4f31-96d5-e0b0b994a3bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:51.785799 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458f81f5-7bba-4f31-96d5-e0b0b994a3bb None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:51.794811 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 262/502] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jul 13 23:01:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:51.795692 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f81f5-7bba-4f31-96d5-e0b0b994a3bb None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:51.858800 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:51.879603 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561e5133-0b2b-4cb0-8ca0-9c9b228b9370 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:51.915239 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a20731d-0d9e-42e7-b009-fd529f4ec531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:51.915940 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20731d-0d9e-42e7-b009-fd529f4ec531 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:51.916224 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20731d-0d9e-42e7-b009-fd529f4ec531 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:51.916501 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a20731d-0d9e-42e7-b009-fd529f4ec531 None admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:51.916718 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f81f5-7bba-4f31-96d5-e0b0b994a3bb None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:51.917362 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a20731d-0d9e-42e7-b009-fd529f4ec531 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:51.917665 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a20731d-0d9e-42e7-b009-fd529f4ec531 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:51.919122 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a20731d-0d9e-42e7-b009-fd529f4ec531 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:51.951553 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26240744-e130-4cac-9a01-bd30f18c24f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:51.952224 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26240744-e130-4cac-9a01-bd30f18c24f2 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:51.952503 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26240744-e130-4cac-9a01-bd30f18c24f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:51.953298 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26240744-e130-4cac-9a01-bd30f18c24f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:51.953528 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26240744-e130-4cac-9a01-bd30f18c24f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:51.953831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26240744-e130-4cac-9a01-bd30f18c24f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:51.955257 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26240744-e130-4cac-9a01-bd30f18c24f2 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:51.957454 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 241/503] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Jul 13 23:01:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:51.958474 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a20731d-0d9e-42e7-b009-fd529f4ec531 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:51.969695 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 263/504] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jul 13 23:01:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:51.970812 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc65c6-bee1-4bd7-a9d8-8d97c2738e5f None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:51.971100 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc65c6-bee1-4bd7-a9d8-8d97c2738e5f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:51.971378 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc65c6-bee1-4bd7-a9d8-8d97c2738e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:51.981484 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:51.988973 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-6cfc65c6-bee1-4bd7-a9d8-8d97c2738e5f None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:51.990200 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6cfc65c6-bee1-4bd7-a9d8-8d97c2738e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:52.060200 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 264/505] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:52.061061 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc65c6-bee1-4bd7-a9d8-8d97c2738e5f None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:52.092205 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a20731d-0d9e-42e7-b009-fd529f4ec531 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:52.107503 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:52.128573 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc159c-78ce-4e51-9f99-2e8a53016df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:52.129245 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc159c-78ce-4e51-9f99-2e8a53016df5 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:52.129531 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc159c-78ce-4e51-9f99-2e8a53016df5 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:52.129801 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cc159c-78ce-4e51-9f99-2e8a53016df5 None admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:52.130653 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cc159c-78ce-4e51-9f99-2e8a53016df5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:52.130961 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cc159c-78ce-4e51-9f99-2e8a53016df5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:52.132458 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cc159c-78ce-4e51-9f99-2e8a53016df5 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:52.153975 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 242/506] 10.222.0.60 () {56 vars in 1137 bytes} [Wed Jul 13 23:01:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:52.154824 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc159c-78ce-4e51-9f99-2e8a53016df5 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:52.171266 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc65c6-bee1-4bd7-a9d8-8d97c2738e5f None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:52.212545 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbef033f-5457-49c3-88f6-41a339a06fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:52.213344 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef033f-5457-49c3-88f6-41a339a06fff None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:52.213625 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef033f-5457-49c3-88f6-41a339a06fff None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:52.213903 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef033f-5457-49c3-88f6-41a339a06fff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:52.224228 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbef033f-5457-49c3-88f6-41a339a06fff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:52.225713 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbef033f-5457-49c3-88f6-41a339a06fff None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:52.233432 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:52.246389 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-dbef033f-5457-49c3-88f6-41a339a06fff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:01:52.246389 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:52.246389 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:52.246389 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:52.246389 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:52.246389 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:52.246389 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:52.246389 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:52.246389 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:52.246389 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:52.246389 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:52.246389 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:52.246389 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:01:52.246389 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:01:52.246389 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:01:52.248225 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:01:52.248225 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:52.248225 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:52.248225 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:52.248225 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:52.248225 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:52.248225 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:52.248225 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:52.248225 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:52.248225 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:52.248225 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:52.248225 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:52.248225 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:52.248225 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:52.248225 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:52.248225 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:52.248225 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:52.249655 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:01:52.249655 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:01:52.249655 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:01:52.249655 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:52.249655 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 265/507] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:01:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:52.249655 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbef033f-5457-49c3-88f6-41a339a06fff None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:52.266548 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cc159c-78ce-4e51-9f99-2e8a53016df5 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:52.302274 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6f54fd-9ebf-41ca-9be4-27fe8a1ccc13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:52.302929 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f54fd-9ebf-41ca-9be4-27fe8a1ccc13 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:52.303210 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f54fd-9ebf-41ca-9be4-27fe8a1ccc13 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:52.303490 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f54fd-9ebf-41ca-9be4-27fe8a1ccc13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:52.353990 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:52.367998 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbef033f-5457-49c3-88f6-41a339a06fff None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:52.378981 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6f54fd-9ebf-41ca-9be4-27fe8a1ccc13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:52.380531 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6f54fd-9ebf-41ca-9be4-27fe8a1ccc13 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:52.403922 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8752c74e-d2c7-47cf-bed1-351c8a4ca49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:52.404719 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752c74e-d2c7-47cf-bed1-351c8a4ca49d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:52.405003 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752c74e-d2c7-47cf-bed1-351c8a4ca49d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:52.405274 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752c74e-d2c7-47cf-bed1-351c8a4ca49d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:52.416299 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8752c74e-d2c7-47cf-bed1-351c8a4ca49d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:52.417787 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8752c74e-d2c7-47cf-bed1-351c8a4ca49d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:52.442042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-8752c74e-d2c7-47cf-bed1-351c8a4ca49d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:01:52.442042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:52.442042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:52.442042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:52.442042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:52.442042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:52.442042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:52.442042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:52.442042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:52.442042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:52.442042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:52.442042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:52.442042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:01:52.442042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:01:52.442042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:01:52.443153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:01:52.443153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:52.443153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:52.443153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:52.443153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:52.443153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:52.443153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:52.443153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:52.443153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:52.443153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:52.443153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:52.443153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:52.443153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:52.443153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:52.443153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:52.443153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:52.443153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:52.444257 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:01:52.444257 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:01:52.444257 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:01:52.444257 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:52.447831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 266/508] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:01:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:52.448628 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8752c74e-d2c7-47cf-bed1-351c8a4ca49d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:52.459615 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:52.484362 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 243/509] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:01:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 23:01:52.485274 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6f54fd-9ebf-41ca-9be4-27fe8a1ccc13 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:52.562590 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8752c74e-d2c7-47cf-bed1-351c8a4ca49d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:52.597022 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cbe066-9977-432d-aa6d-00bedb3b9647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:52.597841 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cbe066-9977-432d-aa6d-00bedb3b9647 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:52.598123 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cbe066-9977-432d-aa6d-00bedb3b9647 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:52.598217 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6f54fd-9ebf-41ca-9be4-27fe8a1ccc13 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:52.598395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cbe066-9977-432d-aa6d-00bedb3b9647 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:52.608420 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cbe066-9977-432d-aa6d-00bedb3b9647 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:52.609856 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cbe066-9977-432d-aa6d-00bedb3b9647 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:52.616615 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:52.627886 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-00cbe066-9977-432d-aa6d-00bedb3b9647 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:01:52.627886 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:52.627886 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:52.627886 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:52.627886 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:52.627886 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:52.627886 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:52.627886 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:52.627886 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:52.627886 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:52.627886 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:52.627886 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:52.627886 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:01:52.627886 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:01:52.627886 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:01:52.628927 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:01:52.628927 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:52.628927 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:52.628927 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:52.628927 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:52.628927 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:52.628927 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:52.628927 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:52.628927 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:52.628927 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:52.628927 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:52.628927 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:52.628927 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:52.628927 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:52.628927 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:52.628927 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:52.628927 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:52.629997 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:01:52.629997 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:01:52.629997 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:01:52.629997 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:52.629997 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 267/510] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:01:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:52.629997 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cbe066-9977-432d-aa6d-00bedb3b9647 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:52.634961 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e1f90c-f06d-4647-9b20-aea7d93ac857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:52.635649 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1f90c-f06d-4647-9b20-aea7d93ac857 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:52.635934 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1f90c-f06d-4647-9b20-aea7d93ac857 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:52.636209 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1f90c-f06d-4647-9b20-aea7d93ac857 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:52.680842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:52.704227 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e1f90c-f06d-4647-9b20-aea7d93ac857 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:52.705748 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e1f90c-f06d-4647-9b20-aea7d93ac857 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:52.730473 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cbe066-9977-432d-aa6d-00bedb3b9647 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:52.771343 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83097e2-037d-42ab-8e09-c242f591cef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:52.772027 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83097e2-037d-42ab-8e09-c242f591cef6 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:52.772305 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83097e2-037d-42ab-8e09-c242f591cef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:52.772578 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83097e2-037d-42ab-8e09-c242f591cef6 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:52.773294 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83097e2-037d-42ab-8e09-c242f591cef6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:52.774154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83097e2-037d-42ab-8e09-c242f591cef6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:52.775684 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83097e2-037d-42ab-8e09-c242f591cef6 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:52.787503 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 268/511] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:52.788317 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83097e2-037d-42ab-8e09-c242f591cef6 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:52.808921 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 244/512] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:01:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 23:01:52.810156 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e1f90c-f06d-4647-9b20-aea7d93ac857 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:52.845737 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:52.891368 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:52.897333 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83097e2-037d-42ab-8e09-c242f591cef6 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:52.914578 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e1f90c-f06d-4647-9b20-aea7d93ac857 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:52.943005 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2d16a-a1a5-4b5f-ae1d-44be20eb0847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:52.943755 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2d16a-a1a5-4b5f-ae1d-44be20eb0847 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:52.944041 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2d16a-a1a5-4b5f-ae1d-44be20eb0847 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:52.944342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2d16a-a1a5-4b5f-ae1d-44be20eb0847 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:52.945070 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a2d16a-a1a5-4b5f-ae1d-44be20eb0847 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:52.946098 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a2d16a-a1a5-4b5f-ae1d-44be20eb0847 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:52.947805 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a2d16a-a1a5-4b5f-ae1d-44be20eb0847 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:52.956278 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92522e48-af1e-4d1f-8da1-2016dc141ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:52.957095 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92522e48-af1e-4d1f-8da1-2016dc141ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:52.957376 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92522e48-af1e-4d1f-8da1-2016dc141ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:52.957649 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92522e48-af1e-4d1f-8da1-2016dc141ac6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:52.959859 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 269/513] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:52.960729 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2d16a-a1a5-4b5f-ae1d-44be20eb0847 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:52.972268 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92522e48-af1e-4d1f-8da1-2016dc141ac6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:52.973841 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92522e48-af1e-4d1f-8da1-2016dc141ac6 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:52.991923 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-92522e48-af1e-4d1f-8da1-2016dc141ac6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:01:52.991923 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:52.991923 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:52.991923 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:52.991923 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:52.991923 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:52.991923 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:52.991923 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:52.991923 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:52.991923 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:52.991923 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:52.991923 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:52.991923 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:01:52.991923 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:01:52.991923 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:01:52.993500 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:01:52.993500 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:52.993500 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:52.993500 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:52.993500 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:52.993500 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:52.993500 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:52.993500 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:52.993500 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:52.993500 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:52.993500 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:52.993500 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:52.993500 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:52.993500 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:52.993500 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:52.993500 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:52.993500 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:52.994542 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:01:52.994542 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:01:52.994542 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:01:52.994542 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:52.994542 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 245/514] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:01:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:52.995028 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92522e48-af1e-4d1f-8da1-2016dc141ac6 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:53.062435 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2d16a-a1a5-4b5f-ae1d-44be20eb0847 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:53.079986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:53.093468 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92522e48-af1e-4d1f-8da1-2016dc141ac6 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:53.097555 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1148b60-a1c1-43c5-b89c-d2eb84e7b0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:53.098214 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1148b60-a1c1-43c5-b89c-d2eb84e7b0ab None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:53.098495 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1148b60-a1c1-43c5-b89c-d2eb84e7b0ab None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:53.098772 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1148b60-a1c1-43c5-b89c-d2eb84e7b0ab None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:53.099495 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1148b60-a1c1-43c5-b89c-d2eb84e7b0ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:53.099831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1148b60-a1c1-43c5-b89c-d2eb84e7b0ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:53.101220 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1148b60-a1c1-43c5-b89c-d2eb84e7b0ab None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:53.113314 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 270/515] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:53.114437 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1148b60-a1c1-43c5-b89c-d2eb84e7b0ab None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:53.121447 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:53.129670 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c838c2-7863-4650-adda-faab7110cd07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:53.130472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c838c2-7863-4650-adda-faab7110cd07 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:53.130757 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c838c2-7863-4650-adda-faab7110cd07 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:53.131032 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c838c2-7863-4650-adda-faab7110cd07 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:53.138916 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c838c2-7863-4650-adda-faab7110cd07 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:53.140424 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c838c2-7863-4650-adda-faab7110cd07 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:53.157773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-66c838c2-7863-4650-adda-faab7110cd07 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 13 23:01:53.157773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:53.157773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:53.157773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:53.157773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:53.157773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:53.157773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:53.157773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:53.157773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:53.157773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:53.157773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:53.157773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:53.157773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 23:01:53.157773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 23:01:53.157773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 23:01:53.157773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 23:01:53.158780 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:53.158780 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:53.158780 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 23:01:53.158780 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:01:53.158780 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 23:01:53.158780 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:01:53.158780 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:53.158780 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:53.158780 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:53.158780 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:53.158780 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:53.158780 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:53.158780 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:53.158780 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:53.158780 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:53.158780 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:53.158780 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:53.158780 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:53.159854 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:53.159854 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:53.159854 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 23:01:53.159854 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 23:01:53.159854 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 23:01:53.159854 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 23:01:53.159854 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 23:01:53.159854 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 23:01:53.159854 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 13 23:01:53.159854 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:53.159854 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 246/516] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jul 13 23:01:52 2022] GET /identity/v3/users/neutron => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:53.159854 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c838c2-7863-4650-adda-faab7110cd07 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:53.218980 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1148b60-a1c1-43c5-b89c-d2eb84e7b0ab None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:53.260025 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7557ff-0dcb-4086-9894-82fe2876fe81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:53.260757 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c838c2-7863-4650-adda-faab7110cd07 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:53.260891 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7557ff-0dcb-4086-9894-82fe2876fe81 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:53.261086 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7557ff-0dcb-4086-9894-82fe2876fe81 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:53.261361 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7557ff-0dcb-4086-9894-82fe2876fe81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:53.272398 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7557ff-0dcb-4086-9894-82fe2876fe81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:53.273857 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7557ff-0dcb-4086-9894-82fe2876fe81 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:53.291490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-1b7557ff-0dcb-4086-9894-82fe2876fe81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:01:53.291490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:53.291490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:53.291490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:53.291490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:53.291490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:53.291490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:53.291490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:53.291490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:53.291490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:53.291490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:53.291490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:53.291490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:01:53.291490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:01:53.291490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:01:53.293226 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:01:53.293226 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:53.293226 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:53.293226 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:53.293226 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:53.293226 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:53.293226 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:53.293226 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:53.293226 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:53.293226 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:53.293226 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:53.293226 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:53.293226 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:53.293226 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:53.293226 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:53.293226 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:53.293226 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:53.294987 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:01:53.294987 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:01:53.294987 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:01:53.294987 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:53.294987 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 271/517] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:01:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:53.294987 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7557ff-0dcb-4086-9894-82fe2876fe81 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:53.309288 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f02133-2d3b-48e0-b408-90cdadd4145d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:53.310676 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f02133-2d3b-48e0-b408-90cdadd4145d None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:53.311291 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f02133-2d3b-48e0-b408-90cdadd4145d None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:53.311733 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f02133-2d3b-48e0-b408-90cdadd4145d None admin] PATH_INFO: `/v3/users/glance` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:53.325338 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f02133-2d3b-48e0-b408-90cdadd4145d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:53.325737 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:53.328065 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f02133-2d3b-48e0-b408-90cdadd4145d None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:53.349306 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-72f02133-2d3b-48e0-b408-90cdadd4145d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 13 23:01:53.349306 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:53.349306 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:53.349306 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:53.349306 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:53.349306 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:53.349306 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:53.349306 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:53.349306 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:53.349306 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:53.349306 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:53.349306 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:53.349306 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 23:01:53.349306 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 23:01:53.349306 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 23:01:53.349306 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 23:01:53.351125 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:53.351125 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:53.351125 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 23:01:53.351125 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:01:53.351125 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 23:01:53.351125 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:01:53.351125 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:53.351125 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:53.351125 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:53.351125 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:53.351125 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:53.351125 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:53.351125 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:53.351125 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:53.351125 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:53.351125 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:53.351125 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:53.351125 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:53.353415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:53.353415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:53.353415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 23:01:53.353415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 23:01:53.353415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 23:01:53.353415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 23:01:53.353415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 23:01:53.353415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 23:01:53.353415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 13 23:01:53.353415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:53.353415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 247/518] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:01:53 2022] GET /identity/v3/users/glance => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:53.353415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f02133-2d3b-48e0-b408-90cdadd4145d None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:53.421310 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7557ff-0dcb-4086-9894-82fe2876fe81 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:53.424285 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:53.454998 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f581cfa-ba8d-40cb-bf50-1d2d6b4cf789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:53.455693 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f581cfa-ba8d-40cb-bf50-1d2d6b4cf789 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:53.455975 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f581cfa-ba8d-40cb-bf50-1d2d6b4cf789 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:53.456250 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f581cfa-ba8d-40cb-bf50-1d2d6b4cf789 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:53.457003 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f581cfa-ba8d-40cb-bf50-1d2d6b4cf789 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:53.457306 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f581cfa-ba8d-40cb-bf50-1d2d6b4cf789 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:53.458732 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f581cfa-ba8d-40cb-bf50-1d2d6b4cf789 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:53.470465 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 272/519] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:01:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:53.471263 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f581cfa-ba8d-40cb-bf50-1d2d6b4cf789 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:53.482197 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f02133-2d3b-48e0-b408-90cdadd4145d None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:53.514978 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2d6d29-72a5-4eb0-b615-479591862888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:53.515700 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d6d29-72a5-4eb0-b615-479591862888 None admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:53.515980 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d6d29-72a5-4eb0-b615-479591862888 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:53.516256 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d6d29-72a5-4eb0-b615-479591862888 None admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:53.516952 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2d6d29-72a5-4eb0-b615-479591862888 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:53.518403 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2d6d29-72a5-4eb0-b615-479591862888 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:53.537859 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 248/520] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Jul 13 23:01:53 2022] POST /identity/v3/users => generated 247 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:01:53.538718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2d6d29-72a5-4eb0-b615-479591862888 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:53.563593 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:53.581338 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f581cfa-ba8d-40cb-bf50-1d2d6b4cf789 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:53.622226 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb61bde-3235-43bc-8a78-22ce91b9dd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:53.623052 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb61bde-3235-43bc-8a78-22ce91b9dd4c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:53.623396 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb61bde-3235-43bc-8a78-22ce91b9dd4c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:53.623770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb61bde-3235-43bc-8a78-22ce91b9dd4c None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:53.624736 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb61bde-3235-43bc-8a78-22ce91b9dd4c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:53.625143 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb61bde-3235-43bc-8a78-22ce91b9dd4c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:53.626844 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb61bde-3235-43bc-8a78-22ce91b9dd4c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:53.654010 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2d6d29-72a5-4eb0-b615-479591862888 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:53.657997 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:53.676204 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 273/521] 10.222.0.60 () {56 vars in 1139 bytes} [Wed Jul 13 23:01:53 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:53.676204 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb61bde-3235-43bc-8a78-22ce91b9dd4c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:53.708358 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b901147-356c-400b-8489-461691d65047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:53.709120 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b901147-356c-400b-8489-461691d65047 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:53.709478 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b901147-356c-400b-8489-461691d65047 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:53.709855 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b901147-356c-400b-8489-461691d65047 None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:53.711462 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b901147-356c-400b-8489-461691d65047 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:53.711744 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b901147-356c-400b-8489-461691d65047 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:53.713256 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b901147-356c-400b-8489-461691d65047 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:53.728902 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 249/522] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:01:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:53.730133 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b901147-356c-400b-8489-461691d65047 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:53.798502 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb61bde-3235-43bc-8a78-22ce91b9dd4c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:53.834218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00b9c51-d17f-400f-879a-caeb023955e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:53.834919 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b9c51-d17f-400f-879a-caeb023955e4 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:53.835196 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b9c51-d17f-400f-879a-caeb023955e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:53.835467 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b9c51-d17f-400f-879a-caeb023955e4 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:53.836874 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00b9c51-d17f-400f-879a-caeb023955e4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:53.837020 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b901147-356c-400b-8489-461691d65047 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:53.837200 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00b9c51-d17f-400f-879a-caeb023955e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:53.838774 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00b9c51-d17f-400f-879a-caeb023955e4 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:53.862479 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 274/523] 10.222.0.60 () {56 vars in 1137 bytes} [Wed Jul 13 23:01:53 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:53.862479 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00b9c51-d17f-400f-879a-caeb023955e4 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:53.866315 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:53.879369 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff145754-ccdb-40fb-bd18-8cd691ac5872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:53.880344 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff145754-ccdb-40fb-bd18-8cd691ac5872 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:53.880725 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff145754-ccdb-40fb-bd18-8cd691ac5872 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:53.881102 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff145754-ccdb-40fb-bd18-8cd691ac5872 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:53.888587 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff145754-ccdb-40fb-bd18-8cd691ac5872 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:53.890303 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff145754-ccdb-40fb-bd18-8cd691ac5872 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:53.906292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-ff145754-ccdb-40fb-bd18-8cd691ac5872 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:01:53.906292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:53.906292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:53.906292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:53.906292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:53.906292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:53.906292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:53.906292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:53.906292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:53.906292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:53.906292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:53.906292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:53.906292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 23:01:53.906292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 23:01:53.906292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 23:01:53.907198 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 23:01:53.907198 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:53.907198 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:53.907198 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:53.907198 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:53.907198 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:53.907198 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:53.907198 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:53.907198 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:53.907198 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:53.907198 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:53.907198 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:53.907198 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:53.907198 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:53.907198 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:53.907198 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:53.907198 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:53.912925 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 23:01:53.912925 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 23:01:53.912925 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 23:01:53.912925 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 23:01:53.912925 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 23:01:53.912925 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 23:01:53.912925 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:01:53.912925 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:53.912925 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 250/524] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:01:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:53.912925 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff145754-ccdb-40fb-bd18-8cd691ac5872 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:53.936980 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:53.984513 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00b9c51-d17f-400f-879a-caeb023955e4 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:54.020167 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89fb784-0c4a-4ca3-a476-41d43caceb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:54.020830 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fb784-0c4a-4ca3-a476-41d43caceb48 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:54.021116 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fb784-0c4a-4ca3-a476-41d43caceb48 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:54.021421 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fb784-0c4a-4ca3-a476-41d43caceb48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:54.021635 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff145754-ccdb-40fb-bd18-8cd691ac5872 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:54.057181 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bf1786-241c-4c82-9fed-c401d21f1bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:54.057972 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf1786-241c-4c82-9fed-c401d21f1bbc None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:54.058257 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf1786-241c-4c82-9fed-c401d21f1bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:54.058527 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf1786-241c-4c82-9fed-c401d21f1bbc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:54.066349 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bf1786-241c-4c82-9fed-c401d21f1bbc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:54.067856 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bf1786-241c-4c82-9fed-c401d21f1bbc None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:54.083055 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-60bf1786-241c-4c82-9fed-c401d21f1bbc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:01:54.083055 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:54.083055 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:54.083055 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:54.083055 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:54.083055 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:54.083055 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:54.083055 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:54.083055 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:54.083055 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:54.083055 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:54.083055 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:54.083055 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 23:01:54.083055 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 23:01:54.083055 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 23:01:54.084201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 23:01:54.084201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:54.084201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:54.084201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:54.084201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:54.084201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:54.084201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:54.084201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:54.084201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:54.084201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:54.084201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:54.084201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:54.084201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:54.084201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:54.084201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:54.084201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:54.084201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:54.085014 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 23:01:54.085014 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 23:01:54.085014 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 23:01:54.085014 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 23:01:54.085014 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 23:01:54.085014 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 23:01:54.085014 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:01:54.085014 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:54.085014 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 251/525] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:01:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:54.085014 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bf1786-241c-4c82-9fed-c401d21f1bbc None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:54.104055 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89fb784-0c4a-4ca3-a476-41d43caceb48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:54.105513 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89fb784-0c4a-4ca3-a476-41d43caceb48 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:54.109093 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:54.177353 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:54.193612 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bf1786-241c-4c82-9fed-c401d21f1bbc None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:54.214223 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 275/526] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:01:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 23:01:54.215450 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89fb784-0c4a-4ca3-a476-41d43caceb48 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:54.233307 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f522e9ea-9649-4613-bbcc-7644ce74354f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:54.234028 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522e9ea-9649-4613-bbcc-7644ce74354f None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:54.234237 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522e9ea-9649-4613-bbcc-7644ce74354f None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:54.234513 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f522e9ea-9649-4613-bbcc-7644ce74354f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:54.312160 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f522e9ea-9649-4613-bbcc-7644ce74354f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:54.313632 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f522e9ea-9649-4613-bbcc-7644ce74354f None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:54.329187 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89fb784-0c4a-4ca3-a476-41d43caceb48 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:54.331642 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:54.370302 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73e3b4-7aa1-48b7-9a47-31da13f94f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:54.370976 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73e3b4-7aa1-48b7-9a47-31da13f94f82 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:54.371258 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73e3b4-7aa1-48b7-9a47-31da13f94f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:54.371561 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73e3b4-7aa1-48b7-9a47-31da13f94f82 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:54.372352 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73e3b4-7aa1-48b7-9a47-31da13f94f82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:54.372658 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73e3b4-7aa1-48b7-9a47-31da13f94f82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:54.374101 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73e3b4-7aa1-48b7-9a47-31da13f94f82 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:54.392596 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 276/527] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jul 13 23:01:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:54.393442 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73e3b4-7aa1-48b7-9a47-31da13f94f82 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:54.414068 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:54.432630 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 252/528] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:01:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 23:01:54.433493 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f522e9ea-9649-4613-bbcc-7644ce74354f None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:54.509445 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73e3b4-7aa1-48b7-9a47-31da13f94f82 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:54.543432 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48d489d-b6e4-4404-b050-7b7e084ca036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:54.543869 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f522e9ea-9649-4613-bbcc-7644ce74354f None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:54.544692 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d489d-b6e4-4404-b050-7b7e084ca036 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:54.545200 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d489d-b6e4-4404-b050-7b7e084ca036 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:54.545647 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48d489d-b6e4-4404-b050-7b7e084ca036 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:54.546633 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48d489d-b6e4-4404-b050-7b7e084ca036 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:54.546937 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48d489d-b6e4-4404-b050-7b7e084ca036 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:54.548445 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48d489d-b6e4-4404-b050-7b7e084ca036 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:54.567357 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 277/529] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jul 13 23:01:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:54.568327 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48d489d-b6e4-4404-b050-7b7e084ca036 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:54.581193 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ee795-8889-4ff8-b8e2-8824a2af2448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:54.581994 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ee795-8889-4ff8-b8e2-8824a2af2448 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:54.582276 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ee795-8889-4ff8-b8e2-8824a2af2448 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:54.582549 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ee795-8889-4ff8-b8e2-8824a2af2448 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:54.593465 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5ee795-8889-4ff8-b8e2-8824a2af2448 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:54.594597 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:54.594943 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5ee795-8889-4ff8-b8e2-8824a2af2448 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:54.613472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-1d5ee795-8889-4ff8-b8e2-8824a2af2448 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:01:54.613472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:54.613472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:54.613472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:54.613472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:54.613472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:54.613472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:54.613472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:54.613472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:54.613472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:54.613472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:54.613472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:54.613472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:01:54.613472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:01:54.613472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:01:54.614375 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:01:54.614375 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:54.614375 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:54.614375 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:54.614375 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:54.614375 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:54.614375 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:54.614375 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:54.614375 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:54.614375 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:54.614375 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:54.614375 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:54.614375 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:54.614375 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:54.614375 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:54.614375 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:54.614375 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:54.615357 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:01:54.615357 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:01:54.615357 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:01:54.615357 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:54.619838 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 253/530] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:01:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:54.620702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ee795-8889-4ff8-b8e2-8824a2af2448 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:54.657517 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:54.682269 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48d489d-b6e4-4404-b050-7b7e084ca036 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:54.722685 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dffd68-4536-47e6-b825-50a6573a9d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:54.723456 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dffd68-4536-47e6-b825-50a6573a9d4d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:54.723764 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dffd68-4536-47e6-b825-50a6573a9d4d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:54.724156 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dffd68-4536-47e6-b825-50a6573a9d4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:54.724994 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dffd68-4536-47e6-b825-50a6573a9d4d None admin] RBAC: Adding query filter params (role.id=92417b14d4984ed9956dcd14f97564f3, scope.project.id=ab0a5581e4e444c293815e1304e20c05, user.id=4789384d158b4995bd30e1e37e30f5f1) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:54.725308 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dffd68-4536-47e6-b825-50a6573a9d4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:54.727276 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dffd68-4536-47e6-b825-50a6573a9d4d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:54.727670 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ee795-8889-4ff8-b8e2-8824a2af2448 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:54.736043 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 278/531] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:01:54 2022] GET /identity/v3/role_assignments?user.id=4789384d158b4995bd30e1e37e30f5f1&scope.project.id=ab0a5581e4e444c293815e1304e20c05&role.id=92417b14d4984ed9956dcd14f97564f3 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:54.737861 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dffd68-4536-47e6-b825-50a6573a9d4d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:54.770835 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff50ed3-cace-4ef5-a3df-52e85fb5454c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:54.771584 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff50ed3-cace-4ef5-a3df-52e85fb5454c None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:54.771942 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff50ed3-cace-4ef5-a3df-52e85fb5454c None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:54.772404 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff50ed3-cace-4ef5-a3df-52e85fb5454c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:54.787601 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff50ed3-cace-4ef5-a3df-52e85fb5454c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:54.790363 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff50ed3-cace-4ef5-a3df-52e85fb5454c None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:54.811870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-5ff50ed3-cace-4ef5-a3df-52e85fb5454c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:01:54.811870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:54.811870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:54.811870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:54.811870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:54.811870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:54.811870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:54.811870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:54.811870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:54.811870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:54.811870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:54.811870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:54.811870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:01:54.811870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:01:54.811870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:01:54.812773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:01:54.812773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:54.812773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:54.812773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:54.812773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:54.812773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:54.812773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:54.812773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:54.812773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:54.812773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:54.812773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:54.812773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:54.812773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:54.812773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:54.812773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:54.812773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:54.812773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:54.814070 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:01:54.814070 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:01:54.814070 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:01:54.814070 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:54.814070 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 254/532] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:01:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:54.814488 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff50ed3-cace-4ef5-a3df-52e85fb5454c None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:54.862187 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:54.880040 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dffd68-4536-47e6-b825-50a6573a9d4d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:54.927946 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f713534-e3c3-4755-901e-31105e788b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:54.929355 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:54.931409 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f713534-e3c3-4755-901e-31105e788b5b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:54.932033 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f713534-e3c3-4755-901e-31105e788b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:54.932505 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f713534-e3c3-4755-901e-31105e788b5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:54.934088 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f713534-e3c3-4755-901e-31105e788b5b None admin] RBAC: Adding query filter params (role.id=92417b14d4984ed9956dcd14f97564f3, scope.project.id=ab0a5581e4e444c293815e1304e20c05, user.id=54661bad914f455cb00d42ec692fde11) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:54.934919 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f713534-e3c3-4755-901e-31105e788b5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:54.937409 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f713534-e3c3-4755-901e-31105e788b5b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:54.943625 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff50ed3-cace-4ef5-a3df-52e85fb5454c None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:54.947227 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 279/533] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:01:54 2022] GET /identity/v3/role_assignments?user.id=54661bad914f455cb00d42ec692fde11&scope.project.id=ab0a5581e4e444c293815e1304e20c05&role.id=92417b14d4984ed9956dcd14f97564f3 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:54.948320 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f713534-e3c3-4755-901e-31105e788b5b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:54.983136 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7864dc-172a-4183-910f-2fa10f7b3680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:54.984101 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7864dc-172a-4183-910f-2fa10f7b3680 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:54.984286 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7864dc-172a-4183-910f-2fa10f7b3680 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:54.984571 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7864dc-172a-4183-910f-2fa10f7b3680 None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:54.985355 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7864dc-172a-4183-910f-2fa10f7b3680 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:54.986038 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7864dc-172a-4183-910f-2fa10f7b3680 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:54.987144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7864dc-172a-4183-910f-2fa10f7b3680 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:55.000963 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 255/534] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:01:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:55.006869 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7864dc-172a-4183-910f-2fa10f7b3680 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:55.158996 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f713534-e3c3-4755-901e-31105e788b5b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:55.230280 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7864dc-172a-4183-910f-2fa10f7b3680 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:55.247036 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f092c1c-f34d-4029-b369-6d3216c3465c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:55.247987 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f092c1c-f34d-4029-b369-6d3216c3465c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:55.248408 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f092c1c-f34d-4029-b369-6d3216c3465c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:55.248939 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f092c1c-f34d-4029-b369-6d3216c3465c None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:55.249986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f092c1c-f34d-4029-b369-6d3216c3465c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:55.250432 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f092c1c-f34d-4029-b369-6d3216c3465c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:55.252061 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f092c1c-f34d-4029-b369-6d3216c3465c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:55.272973 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf608d0-54dd-46c1-8b0f-cede6f6c9703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:55.274348 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf608d0-54dd-46c1-8b0f-cede6f6c9703 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:55.274627 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf608d0-54dd-46c1-8b0f-cede6f6c9703 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:55.274893 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf608d0-54dd-46c1-8b0f-cede6f6c9703 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:55.277891 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 280/535] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:01:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:55.286061 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f092c1c-f34d-4029-b369-6d3216c3465c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:55.289831 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf608d0-54dd-46c1-8b0f-cede6f6c9703 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:55.291426 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf608d0-54dd-46c1-8b0f-cede6f6c9703 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:55.308399 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-6cf608d0-54dd-46c1-8b0f-cede6f6c9703 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:01:55.308399 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:55.308399 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:55.308399 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:55.308399 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:55.308399 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:55.308399 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:55.308399 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:55.308399 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:55.308399 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:55.308399 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:55.308399 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:55.308399 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 23:01:55.308399 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 23:01:55.308399 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 23:01:55.309475 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 23:01:55.309475 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:55.309475 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:55.309475 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:55.309475 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:55.309475 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:55.309475 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:55.309475 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:55.309475 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:55.309475 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:55.309475 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:55.309475 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:55.309475 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:55.309475 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:55.309475 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:55.309475 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:55.309475 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:55.310666 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 23:01:55.310666 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 23:01:55.310666 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 23:01:55.310666 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 23:01:55.310666 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 23:01:55.310666 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 23:01:55.310666 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:01:55.310666 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:55.310666 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 256/536] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:01:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:01:55.315244 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf608d0-54dd-46c1-8b0f-cede6f6c9703 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:55.316223 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:55.342077 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:55.416792 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f092c1c-f34d-4029-b369-6d3216c3465c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:55.454898 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fc42f5-0fb3-43f0-89b3-ce94ae533afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:55.456016 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc42f5-0fb3-43f0-89b3-ce94ae533afd None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:55.456468 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc42f5-0fb3-43f0-89b3-ce94ae533afd None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:55.457101 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc42f5-0fb3-43f0-89b3-ce94ae533afd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:55.465562 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fc42f5-0fb3-43f0-89b3-ce94ae533afd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:55.467360 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fc42f5-0fb3-43f0-89b3-ce94ae533afd None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:55.484478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-16fc42f5-0fb3-43f0-89b3-ce94ae533afd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:01:55.484478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:55.484478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:55.484478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:55.484478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:55.484478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:55.484478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:55.484478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:55.484478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:55.484478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:55.484478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:55.484478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:55.484478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 23:01:55.484478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 23:01:55.484478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 23:01:55.485394 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 23:01:55.485394 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:55.485394 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:55.485394 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:55.485394 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:55.485394 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:55.485394 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:55.485394 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:55.485394 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:55.485394 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:55.485394 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:55.485394 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:55.485394 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:55.485394 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:55.485394 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:55.485394 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:55.485394 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:55.486200 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 23:01:55.486200 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 23:01:55.486200 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 23:01:55.486200 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 23:01:55.486200 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 23:01:55.486200 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 23:01:55.486200 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:01:55.486200 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:55.488888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 281/537] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:01:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:55.496629 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fc42f5-0fb3-43f0-89b3-ce94ae533afd None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:55.542437 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf608d0-54dd-46c1-8b0f-cede6f6c9703 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:55.614317 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:55.619513 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb0533-8757-4c51-9a07-7040cbbec556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:55.620800 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb0533-8757-4c51-9a07-7040cbbec556 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:55.621091 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb0533-8757-4c51-9a07-7040cbbec556 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:55.621371 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb0533-8757-4c51-9a07-7040cbbec556 None admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:55.622304 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bb0533-8757-4c51-9a07-7040cbbec556 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:55.622661 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bb0533-8757-4c51-9a07-7040cbbec556 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:55.624296 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bb0533-8757-4c51-9a07-7040cbbec556 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:55.647181 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fc42f5-0fb3-43f0-89b3-ce94ae533afd None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:55.667660 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 257/538] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jul 13 23:01:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:55.668306 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb0533-8757-4c51-9a07-7040cbbec556 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:55.698837 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c916d1c-bebf-4ca5-9e58-84811542292d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:55.699894 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c916d1c-bebf-4ca5-9e58-84811542292d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:55.700370 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c916d1c-bebf-4ca5-9e58-84811542292d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:55.700855 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c916d1c-bebf-4ca5-9e58-84811542292d None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:55.701898 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c916d1c-bebf-4ca5-9e58-84811542292d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:55.702397 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c916d1c-bebf-4ca5-9e58-84811542292d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:55.704067 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c916d1c-bebf-4ca5-9e58-84811542292d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:55.721525 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 282/539] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jul 13 23:01:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:55.728146 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c916d1c-bebf-4ca5-9e58-84811542292d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:55.797800 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:55.838299 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb0533-8757-4c51-9a07-7040cbbec556 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:55.855092 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c916d1c-bebf-4ca5-9e58-84811542292d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:55.875610 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f9aa24-5553-4246-a0f2-1e6e9e418d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:55.876718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f9aa24-5553-4246-a0f2-1e6e9e418d2c None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:55.877301 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f9aa24-5553-4246-a0f2-1e6e9e418d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:55.877771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f9aa24-5553-4246-a0f2-1e6e9e418d2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:55.878923 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f9aa24-5553-4246-a0f2-1e6e9e418d2c None admin] RBAC: Adding query filter params (role.id=92417b14d4984ed9956dcd14f97564f3, scope.project.id=ab0a5581e4e444c293815e1304e20c05, user.id=2d670df2aff5478ebf61f55358d554de) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:55.879570 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f9aa24-5553-4246-a0f2-1e6e9e418d2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:55.881107 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f9aa24-5553-4246-a0f2-1e6e9e418d2c None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:55.886624 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:55.893292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 258/540] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:01:55 2022] GET /identity/v3/role_assignments?user.id=2d670df2aff5478ebf61f55358d554de&scope.project.id=ab0a5581e4e444c293815e1304e20c05&role.id=92417b14d4984ed9956dcd14f97564f3 => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:55.893292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318b508-6805-497b-8724-18ca76e09d48 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:55.893292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318b508-6805-497b-8724-18ca76e09d48 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:55.893292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318b508-6805-497b-8724-18ca76e09d48 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:55.893292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 259/541] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:55.898993 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0dc22-0adc-457e-9488-0af5b6c2c443 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:55.899308 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0dc22-0adc-457e-9488-0af5b6c2c443 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:55.899596 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0dc22-0adc-457e-9488-0af5b6c2c443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:55.901127 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37160f6b-e733-4d91-a01f-7bf3bee74820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:55.901829 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37160f6b-e733-4d91-a01f-7bf3bee74820 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:55.902111 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37160f6b-e733-4d91-a01f-7bf3bee74820 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:55.902380 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37160f6b-e733-4d91-a01f-7bf3bee74820 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:55.903220 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37160f6b-e733-4d91-a01f-7bf3bee74820 None admin] RBAC: Adding query filter params (role.id=92417b14d4984ed9956dcd14f97564f3, scope.project.id=ab0a5581e4e444c293815e1304e20c05, user.id=2ce9e6497fdb4068b23aa3f7b2c19536) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:55.903527 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37160f6b-e733-4d91-a01f-7bf3bee74820 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:55.905020 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37160f6b-e733-4d91-a01f-7bf3bee74820 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:55.924383 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 283/542] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:01:55 2022] GET /identity/v3/role_assignments?user.id=2ce9e6497fdb4068b23aa3f7b2c19536&scope.project.id=ab0a5581e4e444c293815e1304e20c05&role.id=92417b14d4984ed9956dcd14f97564f3 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:55.924658 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-a4b0dc22-0adc-457e-9488-0af5b6c2c443 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:55.925864 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-a4b0dc22-0adc-457e-9488-0af5b6c2c443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:56.006813 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 260/543] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 23:01:56.045173 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37160f6b-e733-4d91-a01f-7bf3bee74820 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:56.184940 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37160f6b-e733-4d91-a01f-7bf3bee74820 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:56.236264 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b46ab3-79c8-4228-96cb-ee7a2821b6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:56.237062 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b46ab3-79c8-4228-96cb-ee7a2821b6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:56.237341 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b46ab3-79c8-4228-96cb-ee7a2821b6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:56.237612 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b46ab3-79c8-4228-96cb-ee7a2821b6f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:56.255724 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b46ab3-79c8-4228-96cb-ee7a2821b6f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:56.263549 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-d1b46ab3-79c8-4228-96cb-ee7a2821b6f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:01:56.263549 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:56.263549 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:56.263549 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:56.263549 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:56.263549 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:56.263549 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:56.263549 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:56.263549 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:56.263549 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:56.263549 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:56.263549 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:56.263549 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:56.263549 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:56.263549 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:56.263549 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:56.264458 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:56.264458 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:56.264458 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:56.264458 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:56.264458 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:56.264458 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:56.264458 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:56.264458 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:56.264458 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:56.264458 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:56.264458 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:56.264458 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:56.264458 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:56.264458 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:56.264458 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:56.264458 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:56.265239 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:56.265239 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:56.265239 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:56.265239 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:56.265239 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:56.265239 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:56.265239 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:56.265239 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:56.265239 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:56.265239 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:56.265239 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:01:56.265239 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:56.265239 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 284/544] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jul 13 23:01:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:56.271264 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b0dc22-0adc-457e-9488-0af5b6c2c443 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:56.364443 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:56.412086 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b0dc22-0adc-457e-9488-0af5b6c2c443 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:56.463645 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2ed455-0a6e-4d34-9948-99614f4f391a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:56.464522 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ed455-0a6e-4d34-9948-99614f4f391a None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:56.464934 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ed455-0a6e-4d34-9948-99614f4f391a None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:56.465349 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ed455-0a6e-4d34-9948-99614f4f391a None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:56.466472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2ed455-0a6e-4d34-9948-99614f4f391a None admin] RBAC: Adding query filter params (name=service) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:56.466857 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2ed455-0a6e-4d34-9948-99614f4f391a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:56.468649 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2ed455-0a6e-4d34-9948-99614f4f391a None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:56.480311 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 261/545] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jul 13 23:01:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:56.485715 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b46ab3-79c8-4228-96cb-ee7a2821b6f2 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:56.550695 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:56.613768 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b46ab3-79c8-4228-96cb-ee7a2821b6f2 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:56.659785 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547adffa-01a1-41b6-909d-4a97888cd5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:56.660580 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547adffa-01a1-41b6-909d-4a97888cd5ba None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:56.660857 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547adffa-01a1-41b6-909d-4a97888cd5ba None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:56.661128 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547adffa-01a1-41b6-909d-4a97888cd5ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:56.672932 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547adffa-01a1-41b6-909d-4a97888cd5ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:56.674394 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547adffa-01a1-41b6-909d-4a97888cd5ba None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:56.696399 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-547adffa-01a1-41b6-909d-4a97888cd5ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:01:56.696399 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:56.696399 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:56.696399 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:56.696399 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:56.696399 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:56.696399 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:56.696399 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:56.696399 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:56.696399 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:56.696399 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:56.696399 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:56.696399 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:01:56.696399 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:01:56.696399 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:01:56.697589 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:01:56.697589 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:56.697589 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:56.697589 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:56.697589 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:56.697589 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:56.697589 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:56.697589 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:56.697589 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:56.697589 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:56.697589 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:56.697589 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:56.697589 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:56.697589 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:56.697589 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:56.697589 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:56.697589 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:56.698427 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:01:56.698427 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:01:56.698427 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:01:56.698427 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:56.698427 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 285/546] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:01:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:56.703458 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2ed455-0a6e-4d34-9948-99614f4f391a None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:56.836619 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2ed455-0a6e-4d34-9948-99614f4f391a None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:56.861117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:56.872060 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc3066-1758-496c-bd69-fb1058826b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:56.872799 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc3066-1758-496c-bd69-fb1058826b38 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:56.873049 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc3066-1758-496c-bd69-fb1058826b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:56.873362 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc3066-1758-496c-bd69-fb1058826b38 None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:56.874113 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fc3066-1758-496c-bd69-fb1058826b38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:56.874414 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fc3066-1758-496c-bd69-fb1058826b38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:56.875839 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fc3066-1758-496c-bd69-fb1058826b38 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:56.888421 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 262/547] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:01:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:56.896609 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547adffa-01a1-41b6-909d-4a97888cd5ba None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:57.000250 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547adffa-01a1-41b6-909d-4a97888cd5ba None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:57.009921 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:57.035656 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeaf62a-601a-4b7c-8321-7be1cc65dcfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:57.036498 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeaf62a-601a-4b7c-8321-7be1cc65dcfc None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:57.036778 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeaf62a-601a-4b7c-8321-7be1cc65dcfc None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:57.037053 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeaf62a-601a-4b7c-8321-7be1cc65dcfc None admin] PATH_INFO: `/v3/users/swift` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:57.044890 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeaf62a-601a-4b7c-8321-7be1cc65dcfc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:57.046356 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeaf62a-601a-4b7c-8321-7be1cc65dcfc None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:57.061871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-ceeaf62a-601a-4b7c-8321-7be1cc65dcfc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 13 23:01:57.061871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:57.061871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:57.061871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:57.061871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:57.061871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:57.061871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:57.061871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:57.061871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:57.061871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:57.061871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:57.061871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:57.061871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 23:01:57.061871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 23:01:57.061871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 23:01:57.061871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 23:01:57.062820 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:57.062820 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:57.062820 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 23:01:57.062820 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:01:57.062820 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 23:01:57.062820 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:01:57.062820 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:57.062820 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:57.062820 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:57.062820 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:57.062820 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:57.062820 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:57.062820 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:57.062820 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:57.062820 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:57.062820 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:57.062820 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:57.062820 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:57.064400 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:57.064400 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:57.064400 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 23:01:57.064400 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 23:01:57.064400 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 23:01:57.064400 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 23:01:57.064400 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 23:01:57.064400 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 23:01:57.064400 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 13 23:01:57.064400 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:57.064400 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 286/548] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jul 13 23:01:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:57.069005 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc3066-1758-496c-bd69-fb1058826b38 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:57.175621 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc3066-1758-496c-bd69-fb1058826b38 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:57.220854 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c1d47-9488-4df3-aac8-a842d7aa29f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:57.221954 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c1d47-9488-4df3-aac8-a842d7aa29f5 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:57.222444 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c1d47-9488-4df3-aac8-a842d7aa29f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:57.222934 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c1d47-9488-4df3-aac8-a842d7aa29f5 None admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:57.224051 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3c1d47-9488-4df3-aac8-a842d7aa29f5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:57.224605 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3c1d47-9488-4df3-aac8-a842d7aa29f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:57.226288 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3c1d47-9488-4df3-aac8-a842d7aa29f5 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:57.250885 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 263/549] 10.222.0.60 () {56 vars in 1135 bytes} [Wed Jul 13 23:01:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:57.256635 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeaf62a-601a-4b7c-8321-7be1cc65dcfc None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:57.441753 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeaf62a-601a-4b7c-8321-7be1cc65dcfc None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:57.459262 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c23ad8-1845-4387-9cc1-ae6e148ff3b8 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:57.460231 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c23ad8-1845-4387-9cc1-ae6e148ff3b8 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:57.460717 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c23ad8-1845-4387-9cc1-ae6e148ff3b8 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:57.462578 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 264/550] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:57.468228 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a5bcd5-2fbc-4f8d-8cbb-36535020f256 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:57.468782 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a5bcd5-2fbc-4f8d-8cbb-36535020f256 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:57.469266 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a5bcd5-2fbc-4f8d-8cbb-36535020f256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:57.482568 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b566f3f-a88e-4b3c-a6e5-b8117e0a4f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:57.483218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b566f3f-a88e-4b3c-a6e5-b8117e0a4f99 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:57.483494 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b566f3f-a88e-4b3c-a6e5-b8117e0a4f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:57.483819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b566f3f-a88e-4b3c-a6e5-b8117e0a4f99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:57.506302 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-80a5bcd5-2fbc-4f8d-8cbb-36535020f256 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:57.507551 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-80a5bcd5-2fbc-4f8d-8cbb-36535020f256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:57.553689 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:57.598214 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:57.607431 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b566f3f-a88e-4b3c-a6e5-b8117e0a4f99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:57.608982 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b566f3f-a88e-4b3c-a6e5-b8117e0a4f99 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:57.653020 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 265/551] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 23:01:57.688232 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a5bcd5-2fbc-4f8d-8cbb-36535020f256 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:57.736398 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 287/552] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:01:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 13 23:01:57.742955 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b566f3f-a88e-4b3c-a6e5-b8117e0a4f99 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:57.815152 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:57.826352 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a5bcd5-2fbc-4f8d-8cbb-36535020f256 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:57.856544 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b566f3f-a88e-4b3c-a6e5-b8117e0a4f99 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:57.863624 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab44ea8-30a1-4dc4-bcf8-052c07abe518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:57.864507 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab44ea8-30a1-4dc4-bcf8-052c07abe518 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:57.864791 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab44ea8-30a1-4dc4-bcf8-052c07abe518 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:57.865515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab44ea8-30a1-4dc4-bcf8-052c07abe518 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:57.879380 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab44ea8-30a1-4dc4-bcf8-052c07abe518 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:57.890331 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-7ab44ea8-30a1-4dc4-bcf8-052c07abe518 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:01:57.890331 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:57.890331 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:57.890331 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:57.890331 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:57.890331 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:57.890331 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:57.890331 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:57.890331 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:57.890331 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:57.890331 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:57.890331 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:57.890331 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:57.890331 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:57.890331 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:57.890331 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:57.891527 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:57.891527 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:57.891527 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:57.891527 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:57.891527 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:57.891527 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:57.891527 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:57.891527 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:57.891527 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:57.891527 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:57.891527 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:57.891527 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:57.891527 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:57.891527 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:57.891527 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:57.891527 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:57.892347 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:57.892347 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:57.892347 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:57.892347 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:57.892347 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:57.892347 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:57.892347 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:57.892347 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:57.892347 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:57.892347 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:57.892347 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:01:57.892347 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:57.892976 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 266/553] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jul 13 23:01:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:57.894075 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb14c5-b359-4e7d-b977-a324b29f5ea1 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:57.894359 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb14c5-b359-4e7d-b977-a324b29f5ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:57.894634 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb14c5-b359-4e7d-b977-a324b29f5ea1 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:57.899617 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 267/554] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:57.901046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb14c5-b359-4e7d-b977-a324b29f5ea1 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:57.904494 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d07b2de-ea36-4217-b843-4afa56d61552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:57.905417 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d07b2de-ea36-4217-b843-4afa56d61552 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:57.906697 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d07b2de-ea36-4217-b843-4afa56d61552 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:57.906983 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d07b2de-ea36-4217-b843-4afa56d61552 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:57.920224 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d07b2de-ea36-4217-b843-4afa56d61552 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:57.922404 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d07b2de-ea36-4217-b843-4afa56d61552 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:57.940632 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:57.949381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-3d07b2de-ea36-4217-b843-4afa56d61552 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:01:57.949381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:57.949381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:57.949381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:57.949381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:57.949381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:57.949381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:57.949381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:57.949381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:57.949381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:57.949381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:57.949381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:57.949381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:01:57.949381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:01:57.949381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:01:57.950358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:01:57.950358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:57.950358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:57.950358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:57.950358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:57.950358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:57.950358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:57.950358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:57.950358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:57.950358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:57.950358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:57.950358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:57.950358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:57.950358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:57.950358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:57.950358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:57.950358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:57.951870 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:01:57.951870 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:01:57.951870 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:01:57.951870 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:57.951870 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 288/555] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:01:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:57.952313 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e866e2a-7822-4f4f-8042-5f521f017052 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:57.952602 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e866e2a-7822-4f4f-8042-5f521f017052 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:57.952876 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e866e2a-7822-4f4f-8042-5f521f017052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:57.973850 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-2e866e2a-7822-4f4f-8042-5f521f017052 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:57.975086 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-2e866e2a-7822-4f4f-8042-5f521f017052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:58.036794 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb14c5-b359-4e7d-b977-a324b29f5ea1 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:58.079700 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 289/556] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:58.080977 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e866e2a-7822-4f4f-8042-5f521f017052 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:58.097587 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a071afa-d001-4caa-b966-e1fabfa64f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:58.098417 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a071afa-d001-4caa-b966-e1fabfa64f4b None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:58.098631 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a071afa-d001-4caa-b966-e1fabfa64f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:58.098905 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a071afa-d001-4caa-b966-e1fabfa64f4b None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:58.099694 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a071afa-d001-4caa-b966-e1fabfa64f4b None admin] RBAC: Adding query filter params (name=service) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:58.099998 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a071afa-d001-4caa-b966-e1fabfa64f4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:58.101903 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a071afa-d001-4caa-b966-e1fabfa64f4b None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:58.122185 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 268/557] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jul 13 23:01:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:58.122572 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a071afa-d001-4caa-b966-e1fabfa64f4b None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:58.138896 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:58.192973 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e866e2a-7822-4f4f-8042-5f521f017052 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:58.228264 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:58.230159 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0bfd87-7387-4d63-9ea0-4613305593e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:58.230800 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0bfd87-7387-4d63-9ea0-4613305593e5 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:58.231117 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0bfd87-7387-4d63-9ea0-4613305593e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:58.231364 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0bfd87-7387-4d63-9ea0-4613305593e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:58.232130 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0bfd87-7387-4d63-9ea0-4613305593e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:58.232434 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0bfd87-7387-4d63-9ea0-4613305593e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:58.233833 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0bfd87-7387-4d63-9ea0-4613305593e5 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:58.236103 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a071afa-d001-4caa-b966-e1fabfa64f4b None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:58.246035 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 290/558] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:01:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:58.246682 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0bfd87-7387-4d63-9ea0-4613305593e5 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:58.280692 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5b2502-b4e7-431f-b1e4-1510e4191384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:58.281487 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b2502-b4e7-431f-b1e4-1510e4191384 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:58.281786 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b2502-b4e7-431f-b1e4-1510e4191384 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:58.282236 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b2502-b4e7-431f-b1e4-1510e4191384 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:58.294035 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5b2502-b4e7-431f-b1e4-1510e4191384 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:58.301855 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-fc5b2502-b4e7-431f-b1e4-1510e4191384 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:01:58.301855 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:58.301855 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:58.301855 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:58.301855 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:58.301855 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:58.301855 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:58.301855 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:58.301855 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:58.301855 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:58.301855 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:58.301855 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:58.301855 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:58.301855 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:58.301855 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:58.301855 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:58.303243 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:58.303243 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:58.303243 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:58.303243 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:58.303243 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:58.303243 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:58.303243 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:58.303243 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:58.303243 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:58.303243 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:58.303243 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:58.303243 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:58.303243 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:58.303243 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:58.303243 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:58.303243 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:58.304773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:58.304773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:58.304773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:58.304773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:58.304773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:58.304773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:58.304773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:58.304773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:58.304773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:58.304773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:58.304773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:01:58.304773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:58.304773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 269/559] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jul 13 23:01:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:58.304773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5d877-29c0-4a58-b95d-4eef0b0d3637 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:58.305558 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5d877-29c0-4a58-b95d-4eef0b0d3637 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:58.305558 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5d877-29c0-4a58-b95d-4eef0b0d3637 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:58.306365 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 270/560] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:58.310880 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5d877-29c0-4a58-b95d-4eef0b0d3637 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:58.370600 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0bfd87-7387-4d63-9ea0-4613305593e5 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:58.383848 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:58.409117 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd769415-9aba-4c33-bed6-608d67798050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:58.409956 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd769415-9aba-4c33-bed6-608d67798050 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:58.410237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd769415-9aba-4c33-bed6-608d67798050 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:58.410512 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd769415-9aba-4c33-bed6-608d67798050 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:58.422239 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd769415-9aba-4c33-bed6-608d67798050 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:58.423745 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd769415-9aba-4c33-bed6-608d67798050 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:58.424321 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5d877-29c0-4a58-b95d-4eef0b0d3637 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:58.442505 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-bd769415-9aba-4c33-bed6-608d67798050 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:01:58.442505 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:58.442505 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:58.442505 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:58.442505 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:58.442505 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:58.442505 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:58.442505 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:58.442505 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:58.442505 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:58.442505 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:58.442505 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:58.442505 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:01:58.442505 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:01:58.442505 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:01:58.443579 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:01:58.443579 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:58.443579 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:58.443579 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:58.443579 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:58.443579 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:58.443579 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:58.443579 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:58.443579 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:58.443579 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:58.443579 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:58.443579 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:58.443579 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:58.443579 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:58.443579 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:58.443579 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:58.443579 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:58.445195 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:01:58.445195 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:01:58.445195 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:01:58.445195 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:58.445195 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 291/561] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:01:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:58.445195 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd769415-9aba-4c33-bed6-608d67798050 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:58.464519 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b684743-b43d-4074-b1a0-5cc4071fad4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:58.465950 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b684743-b43d-4074-b1a0-5cc4071fad4c None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:58.466233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b684743-b43d-4074-b1a0-5cc4071fad4c None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:58.466504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b684743-b43d-4074-b1a0-5cc4071fad4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:58.473367 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b684743-b43d-4074-b1a0-5cc4071fad4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:58.474827 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b684743-b43d-4074-b1a0-5cc4071fad4c None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:58.490288 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-7b684743-b43d-4074-b1a0-5cc4071fad4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:01:58.490288 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:58.490288 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:58.490288 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:58.490288 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:58.490288 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:58.490288 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:58.490288 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:58.490288 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:58.490288 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:58.490288 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:58.490288 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:58.490288 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 23:01:58.490288 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 23:01:58.490288 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 23:01:58.491223 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 23:01:58.491223 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:58.491223 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:58.491223 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:58.491223 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:58.491223 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:58.491223 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:58.491223 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:58.491223 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:58.491223 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:58.491223 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:58.491223 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:58.491223 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:58.491223 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:58.491223 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:58.491223 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:58.491223 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:58.492261 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 23:01:58.492261 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 23:01:58.492261 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 23:01:58.492261 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 23:01:58.492261 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 23:01:58.492261 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 23:01:58.492261 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:01:58.492261 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:58.492261 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 271/562] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:01:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:58.493658 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a200f23a-dffd-4c9d-8e3e-9c66af1eef84 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:58.493952 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a200f23a-dffd-4c9d-8e3e-9c66af1eef84 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:58.494229 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a200f23a-dffd-4c9d-8e3e-9c66af1eef84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:58.511097 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-a200f23a-dffd-4c9d-8e3e-9c66af1eef84 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:58.512339 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-a200f23a-dffd-4c9d-8e3e-9c66af1eef84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:58.528689 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:58.556265 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd769415-9aba-4c33-bed6-608d67798050 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:58.588528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 272/563] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:58.589383 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a200f23a-dffd-4c9d-8e3e-9c66af1eef84 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:58.590338 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad4fce-e319-4945-bf63-d1fb2ff4bd4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:58.591003 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad4fce-e319-4945-bf63-d1fb2ff4bd4e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:58.591280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad4fce-e319-4945-bf63-d1fb2ff4bd4e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:58.591569 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad4fce-e319-4945-bf63-d1fb2ff4bd4e None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:58.592318 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ad4fce-e319-4945-bf63-d1fb2ff4bd4e None admin] RBAC: Adding query filter params (name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:58.592621 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ad4fce-e319-4945-bf63-d1fb2ff4bd4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:58.593988 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ad4fce-e319-4945-bf63-d1fb2ff4bd4e None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:58.602499 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 292/564] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jul 13 23:01:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:58.605146 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a931f-3690-44a3-abec-89cab641c402 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:58.605477 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a931f-3690-44a3-abec-89cab641c402 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:58.605758 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a931f-3690-44a3-abec-89cab641c402 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:58.607067 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 293/565] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:01:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:01:58.608316 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a931f-3690-44a3-abec-89cab641c402 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:58.635675 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:58.692346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a200f23a-dffd-4c9d-8e3e-9c66af1eef84 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:58.713668 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a931f-3690-44a3-abec-89cab641c402 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:58.728474 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088349c8-f272-45d9-92a6-ba51e1be728c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:58.729152 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088349c8-f272-45d9-92a6-ba51e1be728c None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:58.729433 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088349c8-f272-45d9-92a6-ba51e1be728c None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:58.729707 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088349c8-f272-45d9-92a6-ba51e1be728c None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:58.730440 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088349c8-f272-45d9-92a6-ba51e1be728c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:58.730741 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088349c8-f272-45d9-92a6-ba51e1be728c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:58.732181 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088349c8-f272-45d9-92a6-ba51e1be728c None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:58.743721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 273/566] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:01:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:58.744540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088349c8-f272-45d9-92a6-ba51e1be728c None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:58.746594 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eeaad8-6b21-43d8-b8ba-4519e2a277f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:58.747249 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eeaad8-6b21-43d8-b8ba-4519e2a277f0 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:58.747524 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eeaad8-6b21-43d8-b8ba-4519e2a277f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:58.747820 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eeaad8-6b21-43d8-b8ba-4519e2a277f0 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:58.748867 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eeaad8-6b21-43d8-b8ba-4519e2a277f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:58.749183 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eeaad8-6b21-43d8-b8ba-4519e2a277f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:58.749657 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:58.750625 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eeaad8-6b21-43d8-b8ba-4519e2a277f0 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:58.763900 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 294/567] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jul 13 23:01:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:58.764722 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eeaad8-6b21-43d8-b8ba-4519e2a277f0 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:58.847900 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088349c8-f272-45d9-92a6-ba51e1be728c None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:58.848507 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:58.864593 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eeaad8-6b21-43d8-b8ba-4519e2a277f0 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:58.881920 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff7a2f-7dfd-49bc-b21c-dceff2b38823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:58.883286 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff7a2f-7dfd-49bc-b21c-dceff2b38823 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:58.883637 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff7a2f-7dfd-49bc-b21c-dceff2b38823 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:58.883878 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff7a2f-7dfd-49bc-b21c-dceff2b38823 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:58.894861 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff7a2f-7dfd-49bc-b21c-dceff2b38823 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:58.896361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff7a2f-7dfd-49bc-b21c-dceff2b38823 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:58.900255 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513db8f8-2710-4e8b-91b7-cdafddcab573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:58.901045 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513db8f8-2710-4e8b-91b7-cdafddcab573 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:58.901328 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513db8f8-2710-4e8b-91b7-cdafddcab573 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:58.901596 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513db8f8-2710-4e8b-91b7-cdafddcab573 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:58.911952 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513db8f8-2710-4e8b-91b7-cdafddcab573 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:58.913504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-6dff7a2f-7dfd-49bc-b21c-dceff2b38823 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:01:58.913504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:58.913504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:58.913504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:58.913504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:58.913504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:58.913504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:58.913504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:58.913504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:58.913504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:58.913504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:58.913504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:58.913504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:01:58.913504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:01:58.913504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:01:58.914606 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:01:58.914606 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:58.914606 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:58.914606 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:58.914606 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:58.914606 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:58.914606 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:58.914606 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:58.914606 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:58.914606 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:58.914606 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:58.914606 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:58.914606 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:58.914606 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:58.914606 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:58.914606 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:58.914606 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:58.916027 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:01:58.916027 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:01:58.916027 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:01:58.916027 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:58.916027 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 274/568] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:01:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:58.916585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23436967-cfaa-445d-9417-ea926a63b450 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:58.916972 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23436967-cfaa-445d-9417-ea926a63b450 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:58.917454 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23436967-cfaa-445d-9417-ea926a63b450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:58.919819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-513db8f8-2710-4e8b-91b7-cdafddcab573 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:01:58.919819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:58.919819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:58.919819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:58.919819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:58.919819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:58.919819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:58.919819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:58.919819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:58.919819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:58.919819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:58.919819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:58.919819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:58.919819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:58.919819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:58.919819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:58.922049 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:58.922049 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:58.922049 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:58.922049 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:58.922049 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:58.922049 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:58.922049 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:58.922049 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:58.922049 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:58.922049 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:58.922049 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:58.922049 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:58.922049 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:58.922049 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:58.922049 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:58.922049 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:58.923670 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:58.923670 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:58.923670 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:58.923670 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:58.923670 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:58.923670 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:58.923670 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:58.923670 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:58.923670 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:58.923670 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:58.923670 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:01:58.923670 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:58.923670 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 295/569] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jul 13 23:01:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:58.923670 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513db8f8-2710-4e8b-91b7-cdafddcab573 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:58.947159 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-23436967-cfaa-445d-9417-ea926a63b450 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:01:58.948779 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-23436967-cfaa-445d-9417-ea926a63b450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:01:59.011480 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:59.029677 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 275/570] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:01:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:01:59.030362 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513db8f8-2710-4e8b-91b7-cdafddcab573 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:59.030539 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23436967-cfaa-445d-9417-ea926a63b450 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:59.066872 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f32f3e-e593-4303-8433-df13ac6b5488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:59.067703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f32f3e-e593-4303-8433-df13ac6b5488 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:59.067987 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f32f3e-e593-4303-8433-df13ac6b5488 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:59.068267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f32f3e-e593-4303-8433-df13ac6b5488 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:59.077119 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f32f3e-e593-4303-8433-df13ac6b5488 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:59.078594 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f32f3e-e593-4303-8433-df13ac6b5488 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:59.093101 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-99f32f3e-e593-4303-8433-df13ac6b5488 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 23:01:59.093101 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:59.093101 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:59.093101 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:59.093101 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:59.093101 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:59.093101 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:59.093101 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:59.093101 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:59.093101 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:59.093101 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:59.093101 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:59.093101 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 23:01:59.093101 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 23:01:59.093101 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 23:01:59.093101 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 23:01:59.094144 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:59.094144 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:59.094144 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 23:01:59.094144 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:01:59.094144 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 23:01:59.094144 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:01:59.094144 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:59.094144 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:59.094144 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:59.094144 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:59.094144 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:59.094144 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:59.094144 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:59.094144 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:59.094144 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:59.094144 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:59.094144 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:59.094144 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:59.095059 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:59.095059 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:59.095059 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 23:01:59.095059 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 23:01:59.095059 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 23:01:59.095059 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 23:01:59.095059 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 23:01:59.095059 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 23:01:59.095059 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 23:01:59.095059 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:59.095059 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 296/571] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Jul 13 23:01:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:59.095701 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f32f3e-e593-4303-8433-df13ac6b5488 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:59.099646 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:59.141266 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23436967-cfaa-445d-9417-ea926a63b450 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:59.182425 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d238b-eb94-4e08-b39b-3e3edbcce2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:59.183102 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d238b-eb94-4e08-b39b-3e3edbcce2b2 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:59.183381 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d238b-eb94-4e08-b39b-3e3edbcce2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:59.183690 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d238b-eb94-4e08-b39b-3e3edbcce2b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:59.184577 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7d238b-eb94-4e08-b39b-3e3edbcce2b2 None admin] RBAC: Adding query filter params (role.id=92417b14d4984ed9956dcd14f97564f3, scope.project.id=ab0a5581e4e444c293815e1304e20c05, user.id=786fc44611804d3ca6280ffcab415569) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:59.184885 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7d238b-eb94-4e08-b39b-3e3edbcce2b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:59.186412 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7d238b-eb94-4e08-b39b-3e3edbcce2b2 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:59.192626 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 276/572] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:01:59 2022] GET /identity/v3/role_assignments?user.id=786fc44611804d3ca6280ffcab415569&scope.project.id=ab0a5581e4e444c293815e1304e20c05&role.id=92417b14d4984ed9956dcd14f97564f3 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:59.193515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d238b-eb94-4e08-b39b-3e3edbcce2b2 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:59.212369 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f32f3e-e593-4303-8433-df13ac6b5488 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:59.251770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8944b78-03dd-40ca-ba4a-b8d4b3244b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:59.252473 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8944b78-03dd-40ca-ba4a-b8d4b3244b93 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:59.252733 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8944b78-03dd-40ca-ba4a-b8d4b3244b93 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:59.253005 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8944b78-03dd-40ca-ba4a-b8d4b3244b93 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:59.253773 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8944b78-03dd-40ca-ba4a-b8d4b3244b93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:59.254078 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8944b78-03dd-40ca-ba4a-b8d4b3244b93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:59.255523 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8944b78-03dd-40ca-ba4a-b8d4b3244b93 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:59.261244 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:59.273604 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 297/573] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:01:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:59.273604 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8944b78-03dd-40ca-ba4a-b8d4b3244b93 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:59.328651 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d238b-eb94-4e08-b39b-3e3edbcce2b2 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:59.343881 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:59.378920 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dd1a32-4d9b-40c7-8040-90dfc2b00515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:59.379721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd1a32-4d9b-40c7-8040-90dfc2b00515 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:59.380006 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd1a32-4d9b-40c7-8040-90dfc2b00515 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:59.380278 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd1a32-4d9b-40c7-8040-90dfc2b00515 None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:59.381040 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dd1a32-4d9b-40c7-8040-90dfc2b00515 None admin] RBAC: Adding query filter params (name=service) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:59.381351 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dd1a32-4d9b-40c7-8040-90dfc2b00515 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:59.382805 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dd1a32-4d9b-40c7-8040-90dfc2b00515 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:59.395897 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 277/574] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jul 13 23:01:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:59.395897 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dd1a32-4d9b-40c7-8040-90dfc2b00515 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:59.400385 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8944b78-03dd-40ca-ba4a-b8d4b3244b93 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:59.445532 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda0fcc-b73b-485c-8a68-4619dc631f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:59.446335 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda0fcc-b73b-485c-8a68-4619dc631f2a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:59.446617 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda0fcc-b73b-485c-8a68-4619dc631f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:59.447137 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda0fcc-b73b-485c-8a68-4619dc631f2a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:59.458601 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda0fcc-b73b-485c-8a68-4619dc631f2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:01:59.466321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-5dda0fcc-b73b-485c-8a68-4619dc631f2a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:01:59.466321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:59.466321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:59.466321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:59.466321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:59.466321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:59.466321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:59.466321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:59.466321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:59.466321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:59.466321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:59.466321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:59.466321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:01:59.466321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:01:59.466321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:01:59.466321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:01:59.467280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:01:59.467280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:01:59.467280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:01:59.467280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:01:59.467280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:59.467280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:59.467280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:59.467280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:59.467280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:59.467280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:59.467280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:59.467280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:59.467280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:59.467280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:59.467280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:59.467280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:59.468120 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:59.468120 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:59.468120 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:59.468120 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:59.468120 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:01:59.468120 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:01:59.468120 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:01:59.468120 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:01:59.468120 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:01:59.468120 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:01:59.468120 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:01:59.468120 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:59.468120 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 298/575] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jul 13 23:01:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:59.468814 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda0fcc-b73b-485c-8a68-4619dc631f2a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:59.518905 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dd1a32-4d9b-40c7-8040-90dfc2b00515 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:59.534270 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:59.565106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e332400e-9dd8-403d-a8ab-78159070c8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:59.565983 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332400e-9dd8-403d-a8ab-78159070c8b4 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:59.566357 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332400e-9dd8-403d-a8ab-78159070c8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:59.566729 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332400e-9dd8-403d-a8ab-78159070c8b4 None admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:59.567707 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e332400e-9dd8-403d-a8ab-78159070c8b4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:59.568013 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e332400e-9dd8-403d-a8ab-78159070c8b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:59.569480 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e332400e-9dd8-403d-a8ab-78159070c8b4 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:59.594688 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 278/576] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Jul 13 23:01:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:59.594688 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e332400e-9dd8-403d-a8ab-78159070c8b4 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:59.599793 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda0fcc-b73b-485c-8a68-4619dc631f2a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:59.637335 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:59.651382 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a50d801-069f-4a4a-9b10-53b2555d8afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:59.652356 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a50d801-069f-4a4a-9b10-53b2555d8afe None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:59.652705 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a50d801-069f-4a4a-9b10-53b2555d8afe None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:59.654556 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a50d801-069f-4a4a-9b10-53b2555d8afe None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:59.662888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a50d801-069f-4a4a-9b10-53b2555d8afe None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:59.665463 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a50d801-069f-4a4a-9b10-53b2555d8afe None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:59.681398 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-8a50d801-069f-4a4a-9b10-53b2555d8afe None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 23:01:59.681398 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:59.681398 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:59.681398 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:59.681398 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:59.681398 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:59.681398 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:59.681398 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:59.681398 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:59.681398 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:59.681398 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:59.681398 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:59.681398 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 23:01:59.681398 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 23:01:59.681398 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 23:01:59.681398 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 23:01:59.682607 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:59.682607 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:59.682607 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 23:01:59.682607 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:01:59.682607 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 23:01:59.682607 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:01:59.682607 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:59.682607 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:59.682607 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:59.682607 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:59.682607 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:59.682607 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:59.682607 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:59.682607 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:59.682607 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:59.682607 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:59.682607 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:59.682607 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:59.683798 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:59.683798 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:59.683798 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 23:01:59.683798 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 23:01:59.683798 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 23:01:59.683798 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 23:01:59.683798 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 23:01:59.683798 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 23:01:59.683798 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 23:01:59.683798 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:01:59.684380 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 299/577] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:01:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:59.685412 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a50d801-069f-4a4a-9b10-53b2555d8afe None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:59.713977 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e332400e-9dd8-403d-a8ab-78159070c8b4 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:59.768685 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a6876-41d2-48f2-b185-99719584df20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:59.769863 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a6876-41d2-48f2-b185-99719584df20 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:59.770305 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a6876-41d2-48f2-b185-99719584df20 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:59.770735 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a6876-41d2-48f2-b185-99719584df20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:59.786200 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23a6876-41d2-48f2-b185-99719584df20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:59.788148 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23a6876-41d2-48f2-b185-99719584df20 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:59.815771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-d23a6876-41d2-48f2-b185-99719584df20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:01:59.815771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:01:59.815771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:01:59.815771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:01:59.815771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:01:59.815771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:01:59.815771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:01:59.815771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:01:59.815771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:01:59.815771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:01:59.815771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:01:59.815771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:01:59.815771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:01:59.815771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:01:59.815771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:01:59.817359 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:01:59.817359 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:01:59.817359 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:01:59.817359 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:01:59.817359 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:01:59.817359 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:01:59.817359 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:01:59.817359 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:01:59.817359 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:01:59.817359 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:01:59.817359 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:01:59.817359 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:01:59.817359 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:01:59.817359 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:01:59.817359 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:01:59.817359 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:01:59.817359 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:01:59.818929 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:01:59.818929 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:01:59.818929 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:01:59.818929 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:01:59.818929 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 279/578] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:01:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:01:59.819778 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a6876-41d2-48f2-b185-99719584df20 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:59.831334 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a50d801-069f-4a4a-9b10-53b2555d8afe None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:59.862901 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:59.874460 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d0d72-8c9d-46e3-aff8-b4ec4ce36c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:59.876164 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d0d72-8c9d-46e3-aff8-b4ec4ce36c88 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:59.876547 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d0d72-8c9d-46e3-aff8-b4ec4ce36c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:59.876921 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d0d72-8c9d-46e3-aff8-b4ec4ce36c88 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:59.877774 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d0d72-8c9d-46e3-aff8-b4ec4ce36c88 None admin] RBAC: Adding query filter params (name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:01:59.878179 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d0d72-8c9d-46e3-aff8-b4ec4ce36c88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:59.879670 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d0d72-8c9d-46e3-aff8-b4ec4ce36c88 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:01:59.896194 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 300/579] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jul 13 23:01:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:01:59.896194 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d0d72-8c9d-46e3-aff8-b4ec4ce36c88 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:01:59.942019 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a6876-41d2-48f2-b185-99719584df20 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:01:59.950587 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:01:59.977418 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94777a89-fc84-4370-9619-6cab93a1e3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:01:59.978307 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94777a89-fc84-4370-9619-6cab93a1e3a8 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:01:59.978696 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94777a89-fc84-4370-9619-6cab93a1e3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:01:59.979075 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94777a89-fc84-4370-9619-6cab93a1e3a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:01:59.989868 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94777a89-fc84-4370-9619-6cab93a1e3a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:01:59.991454 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94777a89-fc84-4370-9619-6cab93a1e3a8 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:00.009866 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-94777a89-fc84-4370-9619-6cab93a1e3a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:00.009866 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:00.009866 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:00.009866 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:00.009866 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:00.009866 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:00.009866 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:00.009866 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:00.009866 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:00.009866 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:00.009866 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:00.009866 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:00.009866 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:00.009866 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:00.009866 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:00.010952 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:00.010952 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:00.010952 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:00.010952 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:00.010952 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:00.010952 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:00.010952 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:00.010952 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:00.010952 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:00.010952 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:00.010952 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:00.010952 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:00.010952 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:00.010952 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:00.010952 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:00.010952 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:00.010952 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:00.012115 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:00.012115 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:00.012115 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:00.012115 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:00.012386 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d0d72-8c9d-46e3-aff8-b4ec4ce36c88 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:00.017270 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 280/580] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:01:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:00.017270 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94777a89-fc84-4370-9619-6cab93a1e3a8 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:00.052046 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7831ffeb-4b37-4e3e-bab3-033b1a2aec72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:00.052717 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831ffeb-4b37-4e3e-bab3-033b1a2aec72 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:00.053000 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831ffeb-4b37-4e3e-bab3-033b1a2aec72 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:00.053277 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831ffeb-4b37-4e3e-bab3-033b1a2aec72 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:00.054117 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7831ffeb-4b37-4e3e-bab3-033b1a2aec72 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:00.054467 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7831ffeb-4b37-4e3e-bab3-033b1a2aec72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:00.055966 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7831ffeb-4b37-4e3e-bab3-033b1a2aec72 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:00.076957 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 301/581] 10.222.0.60 () {56 vars in 1137 bytes} [Wed Jul 13 23:01:59 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:00.077792 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7831ffeb-4b37-4e3e-bab3-033b1a2aec72 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:00.102889 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:00.130506 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94777a89-fc84-4370-9619-6cab93a1e3a8 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:00.168445 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59839bf5-7163-4af9-8ad7-61b1ff6267fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:00.169106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59839bf5-7163-4af9-8ad7-61b1ff6267fb None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:00.169450 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59839bf5-7163-4af9-8ad7-61b1ff6267fb None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:00.169787 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59839bf5-7163-4af9-8ad7-61b1ff6267fb None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:00.170575 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59839bf5-7163-4af9-8ad7-61b1ff6267fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:00.170941 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59839bf5-7163-4af9-8ad7-61b1ff6267fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:00.172439 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59839bf5-7163-4af9-8ad7-61b1ff6267fb None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:00.184321 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 281/582] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:00.185315 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59839bf5-7163-4af9-8ad7-61b1ff6267fb None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:00.191669 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7831ffeb-4b37-4e3e-bab3-033b1a2aec72 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:00.195854 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:00.234068 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638541ea-d8df-45c0-878f-34dae1d69936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:00.234857 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638541ea-d8df-45c0-878f-34dae1d69936 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:00.235136 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638541ea-d8df-45c0-878f-34dae1d69936 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:00.235417 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638541ea-d8df-45c0-878f-34dae1d69936 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:00.246659 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638541ea-d8df-45c0-878f-34dae1d69936 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:00.248172 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638541ea-d8df-45c0-878f-34dae1d69936 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:00.266698 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-638541ea-d8df-45c0-878f-34dae1d69936 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:00.266698 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:00.266698 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:00.266698 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:00.266698 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:00.266698 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:00.266698 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:00.266698 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:00.266698 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:00.266698 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:00.266698 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:00.266698 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:00.266698 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:00.266698 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:00.266698 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:00.267865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:00.267865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:00.267865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:00.267865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:00.267865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:00.267865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:00.267865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:00.267865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:00.267865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:00.267865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:00.267865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:00.267865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:00.267865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:00.267865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:00.267865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:00.267865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:00.267865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:00.268778 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:00.268778 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:00.268778 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:00.268778 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:00.268778 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 302/583] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:00.269132 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638541ea-d8df-45c0-878f-34dae1d69936 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:00.302365 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59839bf5-7163-4af9-8ad7-61b1ff6267fb None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:00.341581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093b186-d87d-4566-87e3-e906bceeb1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:00.342635 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093b186-d87d-4566-87e3-e906bceeb1e0 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:00.342916 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093b186-d87d-4566-87e3-e906bceeb1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:00.343182 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093b186-d87d-4566-87e3-e906bceeb1e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:00.343971 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a093b186-d87d-4566-87e3-e906bceeb1e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:00.344274 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a093b186-d87d-4566-87e3-e906bceeb1e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:00.345683 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a093b186-d87d-4566-87e3-e906bceeb1e0 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:00.347462 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:00.367689 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 282/584] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:00.368686 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093b186-d87d-4566-87e3-e906bceeb1e0 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:00.389542 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638541ea-d8df-45c0-878f-34dae1d69936 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:00.424061 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51812de8-72f3-4aac-a03b-3da214ad57a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:00.424786 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51812de8-72f3-4aac-a03b-3da214ad57a8 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:00.425055 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51812de8-72f3-4aac-a03b-3da214ad57a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:00.425360 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51812de8-72f3-4aac-a03b-3da214ad57a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:00.436093 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51812de8-72f3-4aac-a03b-3da214ad57a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:00.437545 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51812de8-72f3-4aac-a03b-3da214ad57a8 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:00.455771 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-51812de8-72f3-4aac-a03b-3da214ad57a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:00.455771 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:00.455771 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:00.455771 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:00.455771 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:00.455771 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:00.455771 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:00.455771 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:00.455771 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:00.455771 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:00.455771 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:00.455771 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:00.455771 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:00.455771 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:00.455771 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:00.456848 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:00.456848 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:00.456848 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:00.456848 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:00.456848 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:00.456848 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:00.456848 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:00.456848 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:00.456848 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:00.456848 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:00.456848 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:00.456848 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:00.456848 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:00.456848 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:00.456848 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:00.456848 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:00.456848 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:00.457865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:00.457865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:00.457865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:00.457865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:00.457865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 303/585] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:00.458212 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51812de8-72f3-4aac-a03b-3da214ad57a8 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:00.472086 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:00.482992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093b186-d87d-4566-87e3-e906bceeb1e0 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:00.519809 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe2aa2-1f12-4cbf-aadf-606cb54c1e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:00.520547 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe2aa2-1f12-4cbf-aadf-606cb54c1e03 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:00.520832 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe2aa2-1f12-4cbf-aadf-606cb54c1e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:00.521111 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe2aa2-1f12-4cbf-aadf-606cb54c1e03 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:00.529277 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbe2aa2-1f12-4cbf-aadf-606cb54c1e03 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:00.530781 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbe2aa2-1f12-4cbf-aadf-606cb54c1e03 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:00.550508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-fbbe2aa2-1f12-4cbf-aadf-606cb54c1e03 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 13 23:02:00.550508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:00.550508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:00.550508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:00.550508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:00.550508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:00.550508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:00.550508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:00.550508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:00.550508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:00.550508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:00.550508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:00.550508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 23:02:00.550508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 23:02:00.550508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 23:02:00.550508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 23:02:00.551510 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:00.551510 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:00.551510 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 23:02:00.551510 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:00.551510 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 23:02:00.551510 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:00.551510 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:00.551510 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:00.551510 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:00.551510 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:00.551510 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:00.551510 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:00.551510 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:00.551510 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:00.551510 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:00.551510 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:00.551510 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:00.551510 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:00.552478 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:00.552478 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:00.552478 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 23:02:00.552478 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 23:02:00.552478 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 23:02:00.552478 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 23:02:00.552478 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 23:02:00.552478 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 23:02:00.552478 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 13 23:02:00.552478 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:00.553011 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 283/586] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:02:00 2022] GET /identity/v3/users/glance => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:00.553697 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe2aa2-1f12-4cbf-aadf-606cb54c1e03 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:00.574200 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51812de8-72f3-4aac-a03b-3da214ad57a8 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:00.611215 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48023392-9b96-49c6-8a10-2155a54bafaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:00.611904 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48023392-9b96-49c6-8a10-2155a54bafaf None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:00.612042 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:00.612181 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48023392-9b96-49c6-8a10-2155a54bafaf None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:00.612455 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48023392-9b96-49c6-8a10-2155a54bafaf None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:00.613175 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48023392-9b96-49c6-8a10-2155a54bafaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:00.613852 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48023392-9b96-49c6-8a10-2155a54bafaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:00.615224 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48023392-9b96-49c6-8a10-2155a54bafaf None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:00.627223 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 304/587] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:00.628070 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48023392-9b96-49c6-8a10-2155a54bafaf None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:00.665584 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe2aa2-1f12-4cbf-aadf-606cb54c1e03 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:00.698929 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:00.702437 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408b4e6-6a1a-4b8f-ba9f-9c39a00f55be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:00.703342 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408b4e6-6a1a-4b8f-ba9f-9c39a00f55be None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:00.703783 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408b4e6-6a1a-4b8f-ba9f-9c39a00f55be None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:00.704134 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408b4e6-6a1a-4b8f-ba9f-9c39a00f55be None admin] PATH_INFO: `/v3/projects/service` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:00.711073 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5408b4e6-6a1a-4b8f-ba9f-9c39a00f55be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:00.712668 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5408b4e6-6a1a-4b8f-ba9f-9c39a00f55be None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:00.727437 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-5408b4e6-6a1a-4b8f-ba9f-9c39a00f55be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:00.727437 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:00.727437 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:00.727437 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:00.727437 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:00.727437 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:00.727437 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:00.727437 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:00.727437 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:00.727437 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:00.727437 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:00.727437 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:00.727437 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 23:02:00.727437 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 23:02:00.727437 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 23:02:00.728498 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 23:02:00.728498 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:00.728498 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:00.728498 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:00.728498 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:00.728498 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:00.728498 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:00.728498 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:00.728498 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:00.728498 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:00.728498 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:00.728498 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:00.728498 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:00.728498 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:00.728498 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:00.728498 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:00.728498 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:00.729594 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 23:02:00.729594 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 23:02:00.729594 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 23:02:00.729594 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 23:02:00.729594 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 23:02:00.729594 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 23:02:00.729594 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:00.729594 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:00.735599 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 284/588] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:02:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:00.735599 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408b4e6-6a1a-4b8f-ba9f-9c39a00f55be None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:00.741444 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48023392-9b96-49c6-8a10-2155a54bafaf None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:00.776531 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed86e85-8c18-4738-b68e-f0b90dd9cf46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:00.777292 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed86e85-8c18-4738-b68e-f0b90dd9cf46 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:00.777568 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed86e85-8c18-4738-b68e-f0b90dd9cf46 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:00.777839 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed86e85-8c18-4738-b68e-f0b90dd9cf46 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:00.778564 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed86e85-8c18-4738-b68e-f0b90dd9cf46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:00.778869 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed86e85-8c18-4738-b68e-f0b90dd9cf46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:00.780287 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed86e85-8c18-4738-b68e-f0b90dd9cf46 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:00.799700 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 305/589] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:00.800555 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed86e85-8c18-4738-b68e-f0b90dd9cf46 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:00.844546 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408b4e6-6a1a-4b8f-ba9f-9c39a00f55be None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:00.860307 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:00.884370 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5e9290-57c6-4eca-a907-0ab9f2177b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:00.885210 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e9290-57c6-4eca-a907-0ab9f2177b1c None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:00.885595 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e9290-57c6-4eca-a907-0ab9f2177b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:00.885972 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e9290-57c6-4eca-a907-0ab9f2177b1c None admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:00.886933 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5e9290-57c6-4eca-a907-0ab9f2177b1c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:00.887353 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5e9290-57c6-4eca-a907-0ab9f2177b1c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:00.888956 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5e9290-57c6-4eca-a907-0ab9f2177b1c None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:00.907277 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed86e85-8c18-4738-b68e-f0b90dd9cf46 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:00.911987 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 285/590] 10.222.0.60 () {56 vars in 1137 bytes} [Wed Jul 13 23:02:00 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:00.911987 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5e9290-57c6-4eca-a907-0ab9f2177b1c None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:00.941722 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:00.951379 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2301560-996c-45f8-9a46-b9368aa5aa2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:00.952209 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2301560-996c-45f8-9a46-b9368aa5aa2b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:00.952715 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2301560-996c-45f8-9a46-b9368aa5aa2b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:00.953100 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2301560-996c-45f8-9a46-b9368aa5aa2b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:00.961153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2301560-996c-45f8-9a46-b9368aa5aa2b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:00.962729 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2301560-996c-45f8-9a46-b9368aa5aa2b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:00.979714 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-d2301560-996c-45f8-9a46-b9368aa5aa2b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 13 23:02:00.979714 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:00.979714 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:00.979714 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:00.979714 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:00.979714 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:00.979714 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:00.979714 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:00.979714 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:00.979714 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:00.979714 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:00.979714 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:00.979714 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 23:02:00.979714 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 23:02:00.979714 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 23:02:00.979714 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 23:02:00.980902 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:00.980902 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:00.980902 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 23:02:00.980902 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:00.980902 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 23:02:00.980902 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:00.980902 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:00.980902 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:00.980902 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:00.980902 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:00.980902 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:00.980902 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:00.980902 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:00.980902 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:00.980902 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:00.980902 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:00.980902 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:00.980902 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:00.982031 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:00.982031 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:00.982031 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 23:02:00.982031 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 23:02:00.982031 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 23:02:00.982031 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 23:02:00.982031 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 23:02:00.982031 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 23:02:00.982031 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 13 23:02:00.982031 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:00.987453 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 306/591] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jul 13 23:02:00 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:00.987453 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2301560-996c-45f8-9a46-b9368aa5aa2b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:01.020520 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5e9290-57c6-4eca-a907-0ab9f2177b1c None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:01.057892 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83009e02-76b0-4744-bc91-fd30341162cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:01.058557 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83009e02-76b0-4744-bc91-fd30341162cc None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:01.058936 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83009e02-76b0-4744-bc91-fd30341162cc None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:01.059112 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83009e02-76b0-4744-bc91-fd30341162cc None admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:01.059948 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83009e02-76b0-4744-bc91-fd30341162cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:01.060257 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83009e02-76b0-4744-bc91-fd30341162cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:01.061875 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83009e02-76b0-4744-bc91-fd30341162cc None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:01.079012 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 286/592] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jul 13 23:02:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:01.079012 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83009e02-76b0-4744-bc91-fd30341162cc None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:01.103324 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2301560-996c-45f8-9a46-b9368aa5aa2b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:01.108855 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:01.147134 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32dad56-2e01-4cf5-a2e9-1965de2a0c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:01.148013 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32dad56-2e01-4cf5-a2e9-1965de2a0c4d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:01.149611 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32dad56-2e01-4cf5-a2e9-1965de2a0c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:01.149611 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32dad56-2e01-4cf5-a2e9-1965de2a0c4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:01.156481 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32dad56-2e01-4cf5-a2e9-1965de2a0c4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:01.157674 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32dad56-2e01-4cf5-a2e9-1965de2a0c4d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:01.173794 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-c32dad56-2e01-4cf5-a2e9-1965de2a0c4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:01.173794 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:01.173794 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:01.173794 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:01.173794 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:01.173794 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:01.173794 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:01.173794 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:01.173794 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:01.173794 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:01.173794 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:01.173794 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:01.173794 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 23:02:01.173794 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 23:02:01.173794 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 23:02:01.174919 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 23:02:01.174919 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:01.174919 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:01.174919 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:01.174919 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:01.174919 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:01.174919 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:01.174919 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:01.174919 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:01.174919 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:01.174919 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:01.174919 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:01.174919 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:01.174919 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:01.174919 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:01.174919 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:01.174919 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:01.180828 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 23:02:01.180828 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 23:02:01.180828 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 23:02:01.180828 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 23:02:01.180828 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 23:02:01.180828 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 23:02:01.180828 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:01.180828 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:01.180828 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 307/593] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:02:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:01.180828 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32dad56-2e01-4cf5-a2e9-1965de2a0c4d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:01.188027 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83009e02-76b0-4744-bc91-fd30341162cc None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:01.222800 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c843ac-623e-41b3-969b-8dfe2a467575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:01.223646 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c843ac-623e-41b3-969b-8dfe2a467575 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:01.223999 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c843ac-623e-41b3-969b-8dfe2a467575 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:01.224344 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c843ac-623e-41b3-969b-8dfe2a467575 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:01.231382 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:01.236550 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c843ac-623e-41b3-969b-8dfe2a467575 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:01.238040 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c843ac-623e-41b3-969b-8dfe2a467575 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:01.255585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-71c843ac-623e-41b3-969b-8dfe2a467575 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:01.255585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:01.255585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:01.255585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:01.255585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:01.255585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:01.255585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:01.255585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:01.255585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:01.255585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:01.255585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:01.255585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:01.255585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:01.255585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:01.255585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:01.256549 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:01.256549 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:01.256549 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:01.256549 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:01.256549 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:01.256549 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:01.256549 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:01.256549 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:01.256549 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:01.256549 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:01.256549 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:01.256549 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:01.256549 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:01.256549 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:01.256549 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:01.256549 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:01.256549 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:01.257463 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:01.257463 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:01.257463 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:01.257463 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:01.261174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 287/594] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:01.262114 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c843ac-623e-41b3-969b-8dfe2a467575 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:01.286691 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32dad56-2e01-4cf5-a2e9-1965de2a0c4d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:01.322786 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d9945-5b9b-4e3d-8d80-63fc7a5f3e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:01.323454 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d9945-5b9b-4e3d-8d80-63fc7a5f3e19 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:01.323770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d9945-5b9b-4e3d-8d80-63fc7a5f3e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:01.324046 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d9945-5b9b-4e3d-8d80-63fc7a5f3e19 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:01.324889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21d9945-5b9b-4e3d-8d80-63fc7a5f3e19 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:01.325202 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21d9945-5b9b-4e3d-8d80-63fc7a5f3e19 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:01.326643 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21d9945-5b9b-4e3d-8d80-63fc7a5f3e19 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:01.344133 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 308/595] 10.222.0.60 () {56 vars in 1139 bytes} [Wed Jul 13 23:02:01 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:01.344953 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d9945-5b9b-4e3d-8d80-63fc7a5f3e19 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:01.353076 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:01.370206 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c843ac-623e-41b3-969b-8dfe2a467575 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:01.404347 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce292411-39ae-4afb-86a5-1a6ee6b8e739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:01.405243 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce292411-39ae-4afb-86a5-1a6ee6b8e739 None admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:01.405529 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce292411-39ae-4afb-86a5-1a6ee6b8e739 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:01.405806 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce292411-39ae-4afb-86a5-1a6ee6b8e739 None admin] PATH_INFO: `/v3/projects/ab0a5581e4e444c293815e1304e20c05/users/4789384d158b4995bd30e1e37e30f5f1/roles/92417b14d4984ed9956dcd14f97564f3` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:01.421155 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce292411-39ae-4afb-86a5-1a6ee6b8e739 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab0a5581e4e444c293815e1304e20c05, user_id=4789384d158b4995bd30e1e37e30f5f1, role_id=92417b14d4984ed9956dcd14f97564f3)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:01.422608 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce292411-39ae-4afb-86a5-1a6ee6b8e739 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:01.434982 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 288/596] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jul 13 23:02:01 2022] PUT /identity/v3/projects/ab0a5581e4e444c293815e1304e20c05/users/4789384d158b4995bd30e1e37e30f5f1/roles/92417b14d4984ed9956dcd14f97564f3 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:02:01.436398 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9613777-8466-44c9-bb24-713c98543db1 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:01.436742 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9613777-8466-44c9-bb24-713c98543db1 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:01.437076 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9613777-8466-44c9-bb24-713c98543db1 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:01.439730 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 289/597] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:01.439730 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9613777-8466-44c9-bb24-713c98543db1 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:01.453011 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d9945-5b9b-4e3d-8d80-63fc7a5f3e19 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:01.462171 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:01.494920 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93f4de4-03d2-4e40-8607-88c322493b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:01.495735 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f4de4-03d2-4e40-8607-88c322493b9d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:01.496110 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f4de4-03d2-4e40-8607-88c322493b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:01.496534 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f4de4-03d2-4e40-8607-88c322493b9d None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:01.497505 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93f4de4-03d2-4e40-8607-88c322493b9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:01.497923 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93f4de4-03d2-4e40-8607-88c322493b9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:01.499681 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93f4de4-03d2-4e40-8607-88c322493b9d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:01.515052 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 309/598] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jul 13 23:02:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:01.516107 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93f4de4-03d2-4e40-8607-88c322493b9d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:01.570964 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9613777-8466-44c9-bb24-713c98543db1 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:01.616014 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d2122-de61-447c-8068-c84efcbda276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:01.616690 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d2122-de61-447c-8068-c84efcbda276 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:01.617098 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d2122-de61-447c-8068-c84efcbda276 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:01.617378 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d2122-de61-447c-8068-c84efcbda276 None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:01.618119 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16d2122-de61-447c-8068-c84efcbda276 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:01.618428 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16d2122-de61-447c-8068-c84efcbda276 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:01.619934 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16d2122-de61-447c-8068-c84efcbda276 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:01.621527 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:01.631990 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93f4de4-03d2-4e40-8607-88c322493b9d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:01.638482 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 290/599] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:01.638482 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34362459-00c6-4deb-822b-0857eb0c60c2 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:01.638482 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34362459-00c6-4deb-822b-0857eb0c60c2 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:01.638482 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34362459-00c6-4deb-822b-0857eb0c60c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:01.653114 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-34362459-00c6-4deb-822b-0857eb0c60c2 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:01.654309 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-34362459-00c6-4deb-822b-0857eb0c60c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:01.672746 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37427c8e-71ed-4ada-bf59-fb8bfb825c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:01.673542 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37427c8e-71ed-4ada-bf59-fb8bfb825c18 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:01.673873 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37427c8e-71ed-4ada-bf59-fb8bfb825c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:01.674145 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37427c8e-71ed-4ada-bf59-fb8bfb825c18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:01.685721 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37427c8e-71ed-4ada-bf59-fb8bfb825c18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:01.687223 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37427c8e-71ed-4ada-bf59-fb8bfb825c18 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:01.704960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-37427c8e-71ed-4ada-bf59-fb8bfb825c18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:01.704960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:01.704960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:01.704960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:01.704960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:01.704960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:01.704960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:01.704960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:01.704960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:01.704960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:01.704960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:01.704960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:01.704960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:01.704960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:01.704960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:01.705900 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:01.705900 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:01.705900 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:01.705900 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:01.705900 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:01.705900 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:01.705900 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:01.705900 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:01.705900 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:01.705900 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:01.705900 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:01.705900 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:01.705900 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:01.705900 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:01.705900 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:01.705900 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:01.705900 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:01.706797 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:01.706797 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:01.706797 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:01.706797 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:01.706797 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 310/600] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:01.707062 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37427c8e-71ed-4ada-bf59-fb8bfb825c18 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:01.728044 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:01.735571 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 291/601] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:02:01.735571 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34362459-00c6-4deb-822b-0857eb0c60c2 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:01.818726 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37427c8e-71ed-4ada-bf59-fb8bfb825c18 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:01.847597 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34362459-00c6-4deb-822b-0857eb0c60c2 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:01.856462 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798d54a-8598-4aad-bceb-3667187fdef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:01.857515 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798d54a-8598-4aad-bceb-3667187fdef4 None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:01.857790 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798d54a-8598-4aad-bceb-3667187fdef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:01.858056 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798d54a-8598-4aad-bceb-3667187fdef4 None admin] PATH_INFO: `/v3/projects/ab0a5581e4e444c293815e1304e20c05/users/54661bad914f455cb00d42ec692fde11/roles/92417b14d4984ed9956dcd14f97564f3` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:01.872159 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:01.875334 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b798d54a-8598-4aad-bceb-3667187fdef4 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab0a5581e4e444c293815e1304e20c05, user_id=54661bad914f455cb00d42ec692fde11, role_id=92417b14d4984ed9956dcd14f97564f3)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:01.876841 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b798d54a-8598-4aad-bceb-3667187fdef4 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:01.886248 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53949e63-16c2-4aa5-b2cb-7c1d5de0f64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:01.887049 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53949e63-16c2-4aa5-b2cb-7c1d5de0f64b None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:01.887323 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53949e63-16c2-4aa5-b2cb-7c1d5de0f64b None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:01.887619 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53949e63-16c2-4aa5-b2cb-7c1d5de0f64b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:01.891650 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 311/602] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jul 13 23:02:01 2022] PUT /identity/v3/projects/ab0a5581e4e444c293815e1304e20c05/users/54661bad914f455cb00d42ec692fde11/roles/92417b14d4984ed9956dcd14f97564f3 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:02:01.892686 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798d54a-8598-4aad-bceb-3667187fdef4 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:01.902095 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53949e63-16c2-4aa5-b2cb-7c1d5de0f64b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:01.903708 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53949e63-16c2-4aa5-b2cb-7c1d5de0f64b None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:01.921206 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-53949e63-16c2-4aa5-b2cb-7c1d5de0f64b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:01.921206 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:01.921206 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:01.921206 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:01.921206 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:01.921206 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:01.921206 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:01.921206 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:01.921206 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:01.921206 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:01.921206 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:01.921206 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:01.921206 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 23:02:01.921206 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 23:02:01.921206 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 23:02:01.922155 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 23:02:01.922155 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:01.922155 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:01.922155 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:01.922155 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:01.922155 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:01.922155 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:01.922155 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:01.922155 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:01.922155 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:01.922155 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:01.922155 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:01.922155 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:01.922155 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:01.922155 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:01.922155 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:01.922155 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:01.923050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 23:02:01.923050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 23:02:01.923050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 23:02:01.923050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 23:02:01.923050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 23:02:01.923050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 23:02:01.923050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:01.923050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:01.923050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 292/603] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:02:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:01.928812 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53949e63-16c2-4aa5-b2cb-7c1d5de0f64b None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:01.981326 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:02.025206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798d54a-8598-4aad-bceb-3667187fdef4 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:02.045615 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53949e63-16c2-4aa5-b2cb-7c1d5de0f64b None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:02.066521 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d55b42-56b3-4f2f-a07d-8218eac028fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:02.067409 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d55b42-56b3-4f2f-a07d-8218eac028fc None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:02.067751 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d55b42-56b3-4f2f-a07d-8218eac028fc None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:02.068090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d55b42-56b3-4f2f-a07d-8218eac028fc None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:02.068945 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d55b42-56b3-4f2f-a07d-8218eac028fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:02.069195 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d55b42-56b3-4f2f-a07d-8218eac028fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:02.071469 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d55b42-56b3-4f2f-a07d-8218eac028fc None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:02.085705 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 312/604] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:02.086551 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d55b42-56b3-4f2f-a07d-8218eac028fc None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:02.086939 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91899a70-14f2-4bcc-93fd-31d5c4b59b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:02.087766 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91899a70-14f2-4bcc-93fd-31d5c4b59b30 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:02.088047 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91899a70-14f2-4bcc-93fd-31d5c4b59b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:02.088318 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91899a70-14f2-4bcc-93fd-31d5c4b59b30 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:02.099604 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91899a70-14f2-4bcc-93fd-31d5c4b59b30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:02:02.107351 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-91899a70-14f2-4bcc-93fd-31d5c4b59b30 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:02:02.107351 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:02.107351 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:02.107351 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:02.107351 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:02.107351 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:02.107351 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:02.107351 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:02.107351 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:02.107351 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:02.107351 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:02.107351 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:02.107351 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:02:02.107351 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:02:02.107351 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:02:02.107351 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:02:02.108346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:02:02.108346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:02:02.108346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:02:02.108346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:02:02.108346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:02.108346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:02.108346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:02.108346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:02.108346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:02.108346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:02.108346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:02.108346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:02.108346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:02.108346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:02.108346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:02.108346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:02.109162 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:02.109162 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:02.109162 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:02.109162 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:02.109162 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:02:02.109162 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:02:02.109162 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:02:02.109162 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:02:02.109162 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:02:02.109162 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:02:02.109162 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:02:02.109162 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:02.109162 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 293/605] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jul 13 23:02:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:02.109858 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91899a70-14f2-4bcc-93fd-31d5c4b59b30 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:02.172475 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:02.213675 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d55b42-56b3-4f2f-a07d-8218eac028fc None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:02.237258 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91899a70-14f2-4bcc-93fd-31d5c4b59b30 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:02.252136 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:02.266253 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5568803d-85d3-43c2-8358-4f919208fe44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:02.267616 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5568803d-85d3-43c2-8358-4f919208fe44 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:02.267904 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5568803d-85d3-43c2-8358-4f919208fe44 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:02.268182 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5568803d-85d3-43c2-8358-4f919208fe44 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:02.269049 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5568803d-85d3-43c2-8358-4f919208fe44 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:02.269529 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5568803d-85d3-43c2-8358-4f919208fe44 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:02.271171 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5568803d-85d3-43c2-8358-4f919208fe44 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:02.288941 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 313/606] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jul 13 23:02:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:02.289822 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5568803d-85d3-43c2-8358-4f919208fe44 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:02.311203 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb6423-e228-441b-addb-29a0abba1573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:02.312724 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb6423-e228-441b-addb-29a0abba1573 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:02.313331 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb6423-e228-441b-addb-29a0abba1573 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:02.313930 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb6423-e228-441b-addb-29a0abba1573 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:02.326212 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeb6423-e228-441b-addb-29a0abba1573 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:02.328924 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeb6423-e228-441b-addb-29a0abba1573 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:02.351411 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-dbeb6423-e228-441b-addb-29a0abba1573 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:02.351411 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:02.351411 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:02.351411 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:02.351411 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:02.351411 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:02.351411 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:02.351411 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:02.351411 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:02.351411 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:02.351411 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:02.351411 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:02.351411 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 23:02:02.351411 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 23:02:02.351411 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 23:02:02.353210 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 23:02:02.353210 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:02.353210 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:02.353210 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:02.353210 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:02.353210 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:02.353210 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:02.353210 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:02.353210 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:02.353210 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:02.353210 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:02.353210 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:02.353210 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:02.353210 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:02.353210 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:02.353210 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:02.353210 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:02.355251 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 23:02:02.355251 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 23:02:02.355251 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 23:02:02.355251 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 23:02:02.355251 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 23:02:02.355251 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 23:02:02.355251 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:02.355251 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:02.355251 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 294/607] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:02:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:02.355251 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb6423-e228-441b-addb-29a0abba1573 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:02.417814 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5568803d-85d3-43c2-8358-4f919208fe44 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:02.453955 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674f241b-e66e-4b4a-b486-3673d1e6ee2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:02.455390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f241b-e66e-4b4a-b486-3673d1e6ee2a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:02.455881 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f241b-e66e-4b4a-b486-3673d1e6ee2a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:02.456358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f241b-e66e-4b4a-b486-3673d1e6ee2a None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:02.457289 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674f241b-e66e-4b4a-b486-3673d1e6ee2a None admin] RBAC: Adding query filter params (name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:02.457872 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674f241b-e66e-4b4a-b486-3673d1e6ee2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:02.459437 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674f241b-e66e-4b4a-b486-3673d1e6ee2a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:02.468479 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 314/608] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jul 13 23:02:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:02.469477 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674f241b-e66e-4b4a-b486-3673d1e6ee2a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:02.473979 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb6423-e228-441b-addb-29a0abba1573 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:02.477043 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:02.511319 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaead12-d1cc-443f-8cb2-0dcd74da36bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:02.512436 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaead12-d1cc-443f-8cb2-0dcd74da36bb None admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:02.512559 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaead12-d1cc-443f-8cb2-0dcd74da36bb None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:02.512764 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaead12-d1cc-443f-8cb2-0dcd74da36bb None admin] PATH_INFO: `/v3/projects/ab0a5581e4e444c293815e1304e20c05/users/2ce9e6497fdb4068b23aa3f7b2c19536/roles/92417b14d4984ed9956dcd14f97564f3` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:02.523224 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:02.528174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaead12-d1cc-443f-8cb2-0dcd74da36bb None admin] RBAC: Authorizing `identity:create_grant(project_id=ab0a5581e4e444c293815e1304e20c05, user_id=2ce9e6497fdb4068b23aa3f7b2c19536, role_id=92417b14d4984ed9956dcd14f97564f3)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:02.529590 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaead12-d1cc-443f-8cb2-0dcd74da36bb None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:02.548304 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 295/609] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jul 13 23:02:02 2022] PUT /identity/v3/projects/ab0a5581e4e444c293815e1304e20c05/users/2ce9e6497fdb4068b23aa3f7b2c19536/roles/92417b14d4984ed9956dcd14f97564f3 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:02:02.549344 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaead12-d1cc-443f-8cb2-0dcd74da36bb None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:02.585989 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674f241b-e66e-4b4a-b486-3673d1e6ee2a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:02.624825 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fada0-d020-4e6d-b752-57bf2a156a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:02.625898 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fada0-d020-4e6d-b752-57bf2a156a2b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:02.626385 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fada0-d020-4e6d-b752-57bf2a156a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:02.626862 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fada0-d020-4e6d-b752-57bf2a156a2b None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:02.627894 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007fada0-d020-4e6d-b752-57bf2a156a2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:02.628435 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007fada0-d020-4e6d-b752-57bf2a156a2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:02.630131 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007fada0-d020-4e6d-b752-57bf2a156a2b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:02.644032 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 315/610] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jul 13 23:02:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:02.653309 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fada0-d020-4e6d-b752-57bf2a156a2b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:02.684695 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaead12-d1cc-443f-8cb2-0dcd74da36bb None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:02.728324 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2675d50e-e0ab-4f45-b5df-d734e8c145b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:02.729141 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675d50e-e0ab-4f45-b5df-d734e8c145b5 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:02.729420 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675d50e-e0ab-4f45-b5df-d734e8c145b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:02.729692 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675d50e-e0ab-4f45-b5df-d734e8c145b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:02.747563 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2675d50e-e0ab-4f45-b5df-d734e8c145b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:02.747563 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2675d50e-e0ab-4f45-b5df-d734e8c145b5 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:02.765595 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-2675d50e-e0ab-4f45-b5df-d734e8c145b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:02.765595 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:02.765595 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:02.765595 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:02.765595 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:02.765595 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:02.765595 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:02.765595 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:02.765595 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:02.765595 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:02.765595 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:02.765595 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:02.765595 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:02.765595 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:02.765595 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:02.766528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:02.766528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:02.766528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:02.766528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:02.766528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:02.766528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:02.766528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:02.766528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:02.766528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:02.766528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:02.766528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:02.766528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:02.766528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:02.766528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:02.766528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:02.766528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:02.766528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:02.767399 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:02.767399 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:02.767399 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:02.767399 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:02.768173 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 296/611] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:02.773820 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2675d50e-e0ab-4f45-b5df-d734e8c145b5 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:02.784251 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:02.834390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fada0-d020-4e6d-b752-57bf2a156a2b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:02.846455 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:02.876233 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6acd61c-925b-4460-b594-d095b5faaf0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:02.877470 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acd61c-925b-4460-b594-d095b5faaf0f None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:02.877470 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acd61c-925b-4460-b594-d095b5faaf0f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:02.877658 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acd61c-925b-4460-b594-d095b5faaf0f None admin] PATH_INFO: `/v3/projects/ab0a5581e4e444c293815e1304e20c05/users/2d670df2aff5478ebf61f55358d554de/roles/92417b14d4984ed9956dcd14f97564f3` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:02.893314 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6acd61c-925b-4460-b594-d095b5faaf0f None admin] RBAC: Authorizing `identity:create_grant(project_id=ab0a5581e4e444c293815e1304e20c05, user_id=2d670df2aff5478ebf61f55358d554de, role_id=92417b14d4984ed9956dcd14f97564f3)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:02.894755 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6acd61c-925b-4460-b594-d095b5faaf0f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:02.897053 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2675d50e-e0ab-4f45-b5df-d734e8c145b5 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:02.912152 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 316/612] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jul 13 23:02:02 2022] PUT /identity/v3/projects/ab0a5581e4e444c293815e1304e20c05/users/2d670df2aff5478ebf61f55358d554de/roles/92417b14d4984ed9956dcd14f97564f3 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:02:02.954121 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986dabb0-680a-47b1-b4af-8d2ccd0b82b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:02.955111 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986dabb0-680a-47b1-b4af-8d2ccd0b82b9 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:02.955623 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986dabb0-680a-47b1-b4af-8d2ccd0b82b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:02.956145 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986dabb0-680a-47b1-b4af-8d2ccd0b82b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:02.957171 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986dabb0-680a-47b1-b4af-8d2ccd0b82b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:02.957628 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986dabb0-680a-47b1-b4af-8d2ccd0b82b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:02.959380 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986dabb0-680a-47b1-b4af-8d2ccd0b82b9 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:02.983307 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 297/613] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:02:02.992790 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6acd61c-925b-4460-b594-d095b5faaf0f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:03.131200 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6acd61c-925b-4460-b594-d095b5faaf0f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:03.168816 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1caf382-e29f-464c-9031-e840e142b312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:03.169668 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1caf382-e29f-464c-9031-e840e142b312 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:03.169941 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1caf382-e29f-464c-9031-e840e142b312 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:03.170261 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1caf382-e29f-464c-9031-e840e142b312 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:03.177994 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1caf382-e29f-464c-9031-e840e142b312 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:03.179697 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1caf382-e29f-464c-9031-e840e142b312 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:03.195584 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-c1caf382-e29f-464c-9031-e840e142b312 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 13 23:02:03.195584 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:03.195584 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:03.195584 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:03.195584 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:03.195584 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:03.195584 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:03.195584 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:03.195584 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:03.195584 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:03.195584 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:03.195584 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:03.195584 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 23:02:03.195584 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 23:02:03.195584 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 23:02:03.195584 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 23:02:03.196557 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:03.196557 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:03.196557 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 23:02:03.196557 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:03.196557 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 23:02:03.196557 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:03.196557 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:03.196557 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:03.196557 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:03.196557 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:03.196557 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:03.196557 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:03.196557 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:03.196557 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:03.196557 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:03.196557 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:03.196557 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:03.196557 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:03.197753 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:03.197753 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:03.197753 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 23:02:03.197753 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 23:02:03.197753 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 23:02:03.197753 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 23:02:03.197753 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 23:02:03.197753 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 23:02:03.197753 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 13 23:02:03.197753 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:03.197753 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 317/614] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jul 13 23:02:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:03.202721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986dabb0-680a-47b1-b4af-8d2ccd0b82b9 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:03.285060 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:03.331569 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986dabb0-680a-47b1-b4af-8d2ccd0b82b9 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:03.383429 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8071d3a-30af-4071-aff5-1c82d8547e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:03.384058 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8071d3a-30af-4071-aff5-1c82d8547e30 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:03.384337 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8071d3a-30af-4071-aff5-1c82d8547e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:03.384609 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8071d3a-30af-4071-aff5-1c82d8547e30 None admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:03.385451 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8071d3a-30af-4071-aff5-1c82d8547e30 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:03.385754 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8071d3a-30af-4071-aff5-1c82d8547e30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:03.387188 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8071d3a-30af-4071-aff5-1c82d8547e30 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:03.415583 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 298/615] 10.222.0.60 () {56 vars in 1135 bytes} [Wed Jul 13 23:02:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:03.420778 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1caf382-e29f-464c-9031-e840e142b312 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:03.453845 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:03.561397 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1caf382-e29f-464c-9031-e840e142b312 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:03.600583 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c65365-9719-4c38-a439-c78953a8c9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:03.601420 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c65365-9719-4c38-a439-c78953a8c9b2 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:03.601698 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c65365-9719-4c38-a439-c78953a8c9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:03.601970 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c65365-9719-4c38-a439-c78953a8c9b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:03.612464 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c65365-9719-4c38-a439-c78953a8c9b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:03.613899 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c65365-9719-4c38-a439-c78953a8c9b2 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:03.632035 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-b5c65365-9719-4c38-a439-c78953a8c9b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:03.632035 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:03.632035 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:03.632035 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:03.632035 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:03.632035 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:03.632035 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:03.632035 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:03.632035 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:03.632035 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:03.632035 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:03.632035 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:03.632035 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:03.632035 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:03.632035 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:03.633770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:03.633770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:03.633770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:03.633770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:03.633770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:03.633770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:03.633770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:03.633770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:03.633770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:03.633770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:03.633770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:03.633770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:03.633770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:03.633770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:03.633770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:03.633770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:03.633770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:03.634680 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:03.634680 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:03.634680 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:03.634680 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:03.638367 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 318/616] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:03.639731 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8071d3a-30af-4071-aff5-1c82d8547e30 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:03.781602 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8071d3a-30af-4071-aff5-1c82d8547e30 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:03.815343 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:03.836228 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159800de-7f31-4495-8ecf-ad1dd69f0ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:03.836903 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159800de-7f31-4495-8ecf-ad1dd69f0ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:03.837183 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159800de-7f31-4495-8ecf-ad1dd69f0ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:03.837456 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159800de-7f31-4495-8ecf-ad1dd69f0ba3 None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:03.838190 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159800de-7f31-4495-8ecf-ad1dd69f0ba3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:03.838492 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159800de-7f31-4495-8ecf-ad1dd69f0ba3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:03.839907 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159800de-7f31-4495-8ecf-ad1dd69f0ba3 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:03.853590 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 299/617] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:03.858354 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c65365-9719-4c38-a439-c78953a8c9b2 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:03.932038 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:03.961312 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c65365-9719-4c38-a439-c78953a8c9b2 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:03.997208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56517f-5a85-4dc5-8145-2a136b49d503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:03.997994 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56517f-5a85-4dc5-8145-2a136b49d503 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:03.998271 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56517f-5a85-4dc5-8145-2a136b49d503 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:03.998544 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56517f-5a85-4dc5-8145-2a136b49d503 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:04.005375 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c56517f-5a85-4dc5-8145-2a136b49d503 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:04.006830 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c56517f-5a85-4dc5-8145-2a136b49d503 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:04.020636 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-0c56517f-5a85-4dc5-8145-2a136b49d503 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:04.020636 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:04.020636 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:04.020636 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:04.020636 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:04.020636 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:04.020636 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:04.020636 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:04.020636 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:04.020636 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:04.020636 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:04.020636 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:04.020636 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 23:02:04.020636 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 23:02:04.020636 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 23:02:04.021649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 23:02:04.021649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:04.021649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:04.021649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:04.021649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:04.021649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:04.021649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:04.021649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:04.021649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:04.021649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:04.021649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:04.021649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:04.021649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:04.021649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:04.021649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:04.021649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:04.021649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:04.022686 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 23:02:04.022686 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 23:02:04.022686 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 23:02:04.022686 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 23:02:04.022686 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 23:02:04.022686 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 23:02:04.022686 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:04.022686 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:04.026790 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 319/618] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:02:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:04.028189 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159800de-7f31-4495-8ecf-ad1dd69f0ba3 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:04.070112 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e63be-4e16-4762-9e6c-fd69bf4b4c13 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:04.070554 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e63be-4e16-4762-9e6c-fd69bf4b4c13 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:04.070864 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e63be-4e16-4762-9e6c-fd69bf4b4c13 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:04.072763 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 320/619] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:04.081288 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11d124c-33a9-41da-8d28-350b097cb070 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:04.081605 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11d124c-33a9-41da-8d28-350b097cb070 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:04.081886 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11d124c-33a9-41da-8d28-350b097cb070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:04.112593 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-d11d124c-33a9-41da-8d28-350b097cb070 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:04.114100 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d11d124c-33a9-41da-8d28-350b097cb070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:04.177783 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159800de-7f31-4495-8ecf-ad1dd69f0ba3 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:04.239946 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 321/620] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:02:04.241082 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772e6298-0627-4895-82e5-11bab33fce76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:04.241791 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e6298-0627-4895-82e5-11bab33fce76 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:04.242075 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e6298-0627-4895-82e5-11bab33fce76 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:04.242349 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e6298-0627-4895-82e5-11bab33fce76 None admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:04.243148 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772e6298-0627-4895-82e5-11bab33fce76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:04.243457 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772e6298-0627-4895-82e5-11bab33fce76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:04.244954 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772e6298-0627-4895-82e5-11bab33fce76 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:04.268849 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11d124c-33a9-41da-8d28-350b097cb070 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:04.272195 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 300/621] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jul 13 23:02:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:04.285042 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772e6298-0627-4895-82e5-11bab33fce76 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:04.339121 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:04.395923 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:04.448132 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11d124c-33a9-41da-8d28-350b097cb070 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:04.487053 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772e6298-0627-4895-82e5-11bab33fce76 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:04.498942 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc45a56-f99d-455c-8cad-51e269bf31d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:04.500063 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc45a56-f99d-455c-8cad-51e269bf31d2 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:04.500421 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc45a56-f99d-455c-8cad-51e269bf31d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:04.500768 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc45a56-f99d-455c-8cad-51e269bf31d2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:04.517982 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc45a56-f99d-455c-8cad-51e269bf31d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:02:04.525179 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283128b9-592f-4f11-82fb-bdec3e1ac4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:04.526094 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283128b9-592f-4f11-82fb-bdec3e1ac4bf None admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:04.526381 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283128b9-592f-4f11-82fb-bdec3e1ac4bf None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:04.526660 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283128b9-592f-4f11-82fb-bdec3e1ac4bf None admin] PATH_INFO: `/v3/projects/ab0a5581e4e444c293815e1304e20c05/users/786fc44611804d3ca6280ffcab415569/roles/92417b14d4984ed9956dcd14f97564f3` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:04.528058 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-5bc45a56-f99d-455c-8cad-51e269bf31d2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:02:04.528058 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:04.528058 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:04.528058 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:04.528058 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:04.528058 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:04.528058 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:04.528058 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:04.528058 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:04.528058 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:04.528058 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:04.528058 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:04.528058 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:02:04.528058 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:02:04.528058 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:02:04.528058 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:02:04.529202 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:02:04.529202 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:02:04.529202 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:02:04.529202 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:02:04.529202 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:04.529202 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:04.529202 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:04.529202 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:04.529202 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:04.529202 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:04.529202 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:04.529202 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:04.529202 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:04.529202 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:04.529202 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:04.529202 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:04.530703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:04.530703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:04.530703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:04.530703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:04.530703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:02:04.530703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:02:04.530703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:02:04.530703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:02:04.530703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:02:04.530703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:02:04.530703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:02:04.530703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:04.531942 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 322/622] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jul 13 23:02:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:02:04.538024 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ded0ef-ae7b-4057-abeb-3712436be633 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:04.538024 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ded0ef-ae7b-4057-abeb-3712436be633 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:04.538024 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ded0ef-ae7b-4057-abeb-3712436be633 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:04.538024 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 323/623] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:04.538024 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ded0ef-ae7b-4057-abeb-3712436be633 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:04.552247 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283128b9-592f-4f11-82fb-bdec3e1ac4bf None admin] RBAC: Authorizing `identity:create_grant(project_id=ab0a5581e4e444c293815e1304e20c05, user_id=786fc44611804d3ca6280ffcab415569, role_id=92417b14d4984ed9956dcd14f97564f3)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:04.553822 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283128b9-592f-4f11-82fb-bdec3e1ac4bf None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:04.568652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 301/624] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jul 13 23:02:04 2022] PUT /identity/v3/projects/ab0a5581e4e444c293815e1304e20c05/users/786fc44611804d3ca6280ffcab415569/roles/92417b14d4984ed9956dcd14f97564f3 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:02:04.570573 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c2174-d02f-43ae-8dbb-9423a2af116a None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:04.570887 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c2174-d02f-43ae-8dbb-9423a2af116a None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:04.571214 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c2174-d02f-43ae-8dbb-9423a2af116a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:04.592711 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-746c2174-d02f-43ae-8dbb-9423a2af116a None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:04.594005 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-746c2174-d02f-43ae-8dbb-9423a2af116a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:04.673962 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 302/625] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:02:04.705459 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c2174-d02f-43ae-8dbb-9423a2af116a None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:04.720910 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ded0ef-ae7b-4057-abeb-3712436be633 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:04.757096 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:04.766450 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:04.790284 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e2b46-d429-4ffe-8ebb-d899fab7cf07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:04.790966 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e2b46-d429-4ffe-8ebb-d899fab7cf07 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:04.791250 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e2b46-d429-4ffe-8ebb-d899fab7cf07 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:04.791524 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e2b46-d429-4ffe-8ebb-d899fab7cf07 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:04.792365 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7e2b46-d429-4ffe-8ebb-d899fab7cf07 None admin] RBAC: Adding query filter params (name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:04.792676 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7e2b46-d429-4ffe-8ebb-d899fab7cf07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:04.794063 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7e2b46-d429-4ffe-8ebb-d899fab7cf07 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:04.808563 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 324/626] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jul 13 23:02:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:02:04.815988 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e2b46-d429-4ffe-8ebb-d899fab7cf07 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:04.881009 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c2174-d02f-43ae-8dbb-9423a2af116a None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:04.939723 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b62bb9-aff2-46db-951f-a01a061e80a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:04.940676 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b62bb9-aff2-46db-951f-a01a061e80a8 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:04.940960 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b62bb9-aff2-46db-951f-a01a061e80a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:04.941236 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b62bb9-aff2-46db-951f-a01a061e80a8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:04.956286 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b62bb9-aff2-46db-951f-a01a061e80a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:02:04.966496 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-b2b62bb9-aff2-46db-951f-a01a061e80a8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:02:04.966496 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:04.966496 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:04.966496 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:04.966496 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:04.966496 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:04.966496 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:04.966496 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:04.966496 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:04.966496 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:04.966496 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:04.966496 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:04.966496 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:02:04.966496 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:02:04.966496 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:02:04.966496 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:02:04.967583 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:02:04.967583 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:02:04.967583 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:02:04.967583 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:02:04.967583 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:04.967583 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:04.967583 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:04.967583 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:04.967583 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:04.967583 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:04.967583 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:04.967583 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:04.967583 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:04.967583 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:04.967583 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:04.967583 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:04.968439 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:04.968439 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:04.968439 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:04.968439 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:04.968439 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:02:04.968439 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:02:04.968439 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:02:04.968439 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:02:04.968439 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:02:04.968439 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:02:04.968439 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:02:04.968439 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:04.968439 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 303/627] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jul 13 23:02:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:02:04.969658 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177d187-36a6-4b7e-8192-0d62e8f4c70a None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:04.969924 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177d187-36a6-4b7e-8192-0d62e8f4c70a None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:04.970362 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7177d187-36a6-4b7e-8192-0d62e8f4c70a None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:04.971070 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e2b46-d429-4ffe-8ebb-d899fab7cf07 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:04.978639 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 304/628] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:04 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:04.983572 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7177d187-36a6-4b7e-8192-0d62e8f4c70a None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:05.025810 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e00d89-3e90-466f-8224-98de810aa513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:05.026973 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e00d89-3e90-466f-8224-98de810aa513 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:05.027581 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e00d89-3e90-466f-8224-98de810aa513 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:05.028098 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e00d89-3e90-466f-8224-98de810aa513 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:05.045068 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e00d89-3e90-466f-8224-98de810aa513 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:05.047200 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e00d89-3e90-466f-8224-98de810aa513 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:05.075598 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-a5e00d89-3e90-466f-8224-98de810aa513 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:05.075598 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:05.075598 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:05.075598 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:05.075598 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:05.075598 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:05.075598 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:05.075598 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:05.075598 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:05.075598 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:05.075598 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:05.075598 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:05.075598 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:05.075598 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:05.075598 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:05.076585 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:05.076585 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:05.076585 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:05.076585 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:05.076585 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:05.076585 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:05.076585 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:05.076585 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:05.076585 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:05.076585 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:05.076585 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:05.076585 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:05.076585 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:05.076585 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:05.076585 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:05.076585 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:05.076585 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:05.077566 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:05.077566 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:05.077566 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:05.077566 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:05.081028 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 325/629] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:05.082609 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44877984-6bc9-433c-9691-3251860ba734 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:05.083210 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44877984-6bc9-433c-9691-3251860ba734 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:05.083771 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44877984-6bc9-433c-9691-3251860ba734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:05.104619 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-44877984-6bc9-433c-9691-3251860ba734 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:05.105826 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-44877984-6bc9-433c-9691-3251860ba734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:05.124463 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:05.141782 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:05.148118 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7177d187-36a6-4b7e-8192-0d62e8f4c70a None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:05.192899 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 326/630] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:02:05.192899 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44877984-6bc9-433c-9691-3251860ba734 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:05.196320 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f666b1b5-8558-4896-8e14-27bea7f4e708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:05.197701 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666b1b5-8558-4896-8e14-27bea7f4e708 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:05.198209 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666b1b5-8558-4896-8e14-27bea7f4e708 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:05.198741 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666b1b5-8558-4896-8e14-27bea7f4e708 None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:05.199851 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f666b1b5-8558-4896-8e14-27bea7f4e708 None admin] RBAC: Adding query filter params (name=service) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:05.200467 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f666b1b5-8558-4896-8e14-27bea7f4e708 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:05.202364 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f666b1b5-8558-4896-8e14-27bea7f4e708 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:05.220683 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 305/631] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jul 13 23:02:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:05.227309 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f666b1b5-8558-4896-8e14-27bea7f4e708 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:05.338845 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44877984-6bc9-433c-9691-3251860ba734 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:05.375593 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f666b1b5-8558-4896-8e14-27bea7f4e708 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:05.394300 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b447e9a9-be10-4126-98dc-60b8ce608fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:05.394981 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447e9a9-be10-4126-98dc-60b8ce608fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:05.395261 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447e9a9-be10-4126-98dc-60b8ce608fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:05.395975 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447e9a9-be10-4126-98dc-60b8ce608fa5 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:05.396316 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b447e9a9-be10-4126-98dc-60b8ce608fa5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:05.396621 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b447e9a9-be10-4126-98dc-60b8ce608fa5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:05.398093 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b447e9a9-be10-4126-98dc-60b8ce608fa5 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:05.415671 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 327/632] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:05.416647 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b447e9a9-be10-4126-98dc-60b8ce608fa5 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:05.424412 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e5db2b-16fb-48bf-b339-ad46c4722397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:05.425590 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5db2b-16fb-48bf-b339-ad46c4722397 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:05.426055 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5db2b-16fb-48bf-b339-ad46c4722397 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:05.426506 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5db2b-16fb-48bf-b339-ad46c4722397 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:05.443838 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e5db2b-16fb-48bf-b339-ad46c4722397 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:05.446337 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e5db2b-16fb-48bf-b339-ad46c4722397 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:05.450221 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:05.460545 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:05.482325 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-b6e5db2b-16fb-48bf-b339-ad46c4722397 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:05.482325 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:05.482325 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:05.482325 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:05.482325 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:05.482325 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:05.482325 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:05.482325 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:05.482325 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:05.482325 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:05.482325 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:05.482325 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:05.482325 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:05.482325 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:05.482325 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:05.483299 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:05.483299 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:05.483299 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:05.483299 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:05.483299 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:05.483299 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:05.483299 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:05.483299 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:05.483299 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:05.483299 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:05.483299 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:05.483299 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:05.483299 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:05.483299 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:05.483299 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:05.483299 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:05.483299 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:05.484228 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:05.484228 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:05.484228 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:05.484228 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:05.490161 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 306/633] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:05.490161 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e5db2b-16fb-48bf-b339-ad46c4722397 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:05.557504 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b447e9a9-be10-4126-98dc-60b8ce608fa5 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:05.600285 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7fd12-050c-4e0a-8607-a045a12ad3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:05.601640 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7fd12-050c-4e0a-8607-a045a12ad3b1 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:05.601987 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7fd12-050c-4e0a-8607-a045a12ad3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:05.602332 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7fd12-050c-4e0a-8607-a045a12ad3b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:05.602466 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e5db2b-16fb-48bf-b339-ad46c4722397 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:05.613067 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c7fd12-050c-4e0a-8607-a045a12ad3b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:02:05.621559 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-61c7fd12-050c-4e0a-8607-a045a12ad3b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:02:05.621559 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:05.621559 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:05.621559 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:05.621559 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:05.621559 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:05.621559 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:05.621559 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:05.621559 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:05.621559 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:05.621559 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:05.621559 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:05.621559 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:02:05.621559 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:02:05.621559 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:02:05.621559 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:02:05.622490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:02:05.622490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:02:05.622490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:02:05.622490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:02:05.622490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:05.622490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:05.622490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:05.622490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:05.622490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:05.622490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:05.622490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:05.622490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:05.622490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:05.622490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:05.622490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:05.622490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:05.628357 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:05.628357 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:05.628357 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:05.628357 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:05.628357 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:02:05.628357 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:02:05.628357 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:02:05.628357 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:02:05.628357 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:02:05.628357 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:02:05.628357 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:02:05.628357 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:05.628357 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 328/634] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jul 13 23:02:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:05.628357 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7fd12-050c-4e0a-8607-a045a12ad3b1 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:05.647364 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee406a-054e-45d9-b1b6-069128353b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:05.648733 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee406a-054e-45d9-b1b6-069128353b8f None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:05.649013 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee406a-054e-45d9-b1b6-069128353b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:05.649284 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee406a-054e-45d9-b1b6-069128353b8f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:05.658060 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ee406a-054e-45d9-b1b6-069128353b8f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:05.659517 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ee406a-054e-45d9-b1b6-069128353b8f None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:05.674718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-56ee406a-054e-45d9-b1b6-069128353b8f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 23:02:05.674718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:05.674718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:05.674718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:05.674718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:05.674718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:05.674718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:05.674718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:05.674718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:05.674718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:05.674718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:05.674718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:05.674718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 23:02:05.674718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 23:02:05.674718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 23:02:05.674718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 23:02:05.675740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:05.675740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:05.675740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 23:02:05.675740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:05.675740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 23:02:05.675740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:05.675740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:05.675740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:05.675740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:05.675740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:05.675740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:05.675740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:05.675740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:05.675740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:05.675740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:05.675740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:05.675740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:05.675740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:05.676660 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:05.676660 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:05.676660 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 23:02:05.676660 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 23:02:05.676660 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 23:02:05.676660 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 23:02:05.676660 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 23:02:05.676660 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 23:02:05.676660 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 23:02:05.676660 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:05.679849 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 307/635] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Jul 13 23:02:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:05.681484 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190f289-a0a1-41d5-8b0d-8df3d84a8177 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:05.681769 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190f289-a0a1-41d5-8b0d-8df3d84a8177 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:05.682043 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190f289-a0a1-41d5-8b0d-8df3d84a8177 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:05.683348 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 308/636] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:05.684052 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190f289-a0a1-41d5-8b0d-8df3d84a8177 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:05.735280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:05.751673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7fd12-050c-4e0a-8607-a045a12ad3b1 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:05.752992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:05.789458 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89beb30c-a700-4969-b30a-eb23aa05d413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:05.790570 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89beb30c-a700-4969-b30a-eb23aa05d413 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:05.790854 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89beb30c-a700-4969-b30a-eb23aa05d413 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:05.791190 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89beb30c-a700-4969-b30a-eb23aa05d413 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:05.791903 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89beb30c-a700-4969-b30a-eb23aa05d413 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:05.792218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89beb30c-a700-4969-b30a-eb23aa05d413 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:05.793594 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89beb30c-a700-4969-b30a-eb23aa05d413 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:05.803567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190f289-a0a1-41d5-8b0d-8df3d84a8177 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:05.811379 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 329/637] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:05.811379 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89beb30c-a700-4969-b30a-eb23aa05d413 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:05.839788 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349664d4-21f4-41ae-a722-be03a921d688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:05.840477 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349664d4-21f4-41ae-a722-be03a921d688 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:05.840761 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349664d4-21f4-41ae-a722-be03a921d688 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:05.841035 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349664d4-21f4-41ae-a722-be03a921d688 None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:05.841816 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349664d4-21f4-41ae-a722-be03a921d688 None admin] RBAC: Adding query filter params (name=service) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:05.842117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349664d4-21f4-41ae-a722-be03a921d688 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:05.843620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349664d4-21f4-41ae-a722-be03a921d688 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:05.857874 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 309/638] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jul 13 23:02:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:05.857874 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fd4ef-e94c-4d24-b100-a24ee516874b None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:05.857874 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fd4ef-e94c-4d24-b100-a24ee516874b None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:05.857874 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fd4ef-e94c-4d24-b100-a24ee516874b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:05.872316 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-0d2fd4ef-e94c-4d24-b100-a24ee516874b None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:05.873525 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-0d2fd4ef-e94c-4d24-b100-a24ee516874b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:05.924567 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89beb30c-a700-4969-b30a-eb23aa05d413 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:05.955561 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 310/639] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:02:05.955561 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2fd4ef-e94c-4d24-b100-a24ee516874b None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:05.975194 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad7c652-2c31-40b1-a1f4-e80d24b69bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:05.977145 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad7c652-2c31-40b1-a1f4-e80d24b69bde None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:05.977579 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad7c652-2c31-40b1-a1f4-e80d24b69bde None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:05.977858 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad7c652-2c31-40b1-a1f4-e80d24b69bde None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:05.978708 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad7c652-2c31-40b1-a1f4-e80d24b69bde None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:05.979015 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad7c652-2c31-40b1-a1f4-e80d24b69bde None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:05.980622 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad7c652-2c31-40b1-a1f4-e80d24b69bde None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:05.998986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 330/640] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Jul 13 23:02:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:06.000429 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad7c652-2c31-40b1-a1f4-e80d24b69bde None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:06.008202 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:06.014107 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:06.069283 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2fd4ef-e94c-4d24-b100-a24ee516874b None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:06.105038 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f287729-cfdd-4581-80ec-6933388d7c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:06.105852 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f287729-cfdd-4581-80ec-6933388d7c28 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:06.106135 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f287729-cfdd-4581-80ec-6933388d7c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:06.106413 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f287729-cfdd-4581-80ec-6933388d7c28 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:06.109749 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad7c652-2c31-40b1-a1f4-e80d24b69bde None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:06.113928 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f287729-cfdd-4581-80ec-6933388d7c28 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:06.115395 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f287729-cfdd-4581-80ec-6933388d7c28 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:06.130116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-4f287729-cfdd-4581-80ec-6933388d7c28 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 23:02:06.130116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:06.130116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:06.130116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:06.130116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:06.130116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:06.130116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:06.130116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:06.130116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:06.130116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:06.130116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:06.130116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:06.130116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 23:02:06.130116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 23:02:06.130116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 23:02:06.130116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 23:02:06.131072 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:06.131072 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:06.131072 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 23:02:06.131072 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:06.131072 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 23:02:06.131072 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:06.131072 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:06.131072 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:06.131072 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:06.131072 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:06.131072 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:06.131072 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:06.131072 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:06.131072 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:06.131072 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:06.131072 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:06.131072 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:06.131072 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:06.132046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:06.132046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:06.132046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 23:02:06.132046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 23:02:06.132046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 23:02:06.132046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 23:02:06.132046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 23:02:06.132046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 23:02:06.132046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 23:02:06.132046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:06.132046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 311/641] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:02:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:06.133180 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f287729-cfdd-4581-80ec-6933388d7c28 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:06.149571 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572fbae-5ffc-46de-8db5-cad285b6c8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:06.150358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572fbae-5ffc-46de-8db5-cad285b6c8eb None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:06.150635 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572fbae-5ffc-46de-8db5-cad285b6c8eb None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:06.150901 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572fbae-5ffc-46de-8db5-cad285b6c8eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:06.161048 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5572fbae-5ffc-46de-8db5-cad285b6c8eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:06.162486 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5572fbae-5ffc-46de-8db5-cad285b6c8eb None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:06.180538 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-5572fbae-5ffc-46de-8db5-cad285b6c8eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:06.180538 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:06.180538 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:06.180538 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:06.180538 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:06.180538 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:06.180538 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:06.180538 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:06.180538 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:06.180538 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:06.180538 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:06.180538 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:06.180538 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:06.180538 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:06.180538 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:06.181537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:06.181537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:06.181537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:06.181537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:06.181537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:06.181537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:06.181537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:06.181537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:06.181537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:06.181537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:06.181537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:06.181537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:06.181537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:06.181537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:06.181537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:06.181537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:06.181537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:06.182409 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:06.182409 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:06.182409 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:06.182409 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:06.182409 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 331/642] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:06.182689 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572fbae-5ffc-46de-8db5-cad285b6c8eb None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:06.238839 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:06.242686 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f287729-cfdd-4581-80ec-6933388d7c28 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:06.254983 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:06.282611 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2dadaa-864f-4f62-b1c0-24454691edfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:06.283686 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2dadaa-864f-4f62-b1c0-24454691edfe None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:06.284369 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2dadaa-864f-4f62-b1c0-24454691edfe None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:06.284649 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2dadaa-864f-4f62-b1c0-24454691edfe None admin] PATH_INFO: `/v3/roles/service` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:06.292960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572fbae-5ffc-46de-8db5-cad285b6c8eb None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:06.295266 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2dadaa-864f-4f62-b1c0-24454691edfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:02:06.302874 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-3e2dadaa-864f-4f62-b1c0-24454691edfe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:02:06.302874 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:06.302874 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:06.302874 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:06.302874 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:06.302874 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:06.302874 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:06.302874 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:06.302874 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:06.302874 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:06.302874 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:06.302874 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:06.302874 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:02:06.302874 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:02:06.302874 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:02:06.302874 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:02:06.303842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:02:06.303842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:02:06.303842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:02:06.303842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:02:06.303842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:06.303842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:06.303842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:06.303842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:06.303842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:06.303842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:06.303842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:06.303842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:06.303842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:06.303842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:06.303842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:06.303842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:06.304721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:06.304721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:06.304721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:06.304721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:06.304721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:02:06.304721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:02:06.304721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:02:06.304721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:02:06.304721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:02:06.304721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:02:06.304721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:02:06.304721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:06.309355 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 312/643] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jul 13 23:02:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:06.309726 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2dadaa-864f-4f62-b1c0-24454691edfe None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:06.330495 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc31339-f8dc-4ccf-a48a-cef2de88122f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:06.331154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc31339-f8dc-4ccf-a48a-cef2de88122f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:06.331435 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc31339-f8dc-4ccf-a48a-cef2de88122f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:06.331741 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc31339-f8dc-4ccf-a48a-cef2de88122f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:06.410480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc31339-f8dc-4ccf-a48a-cef2de88122f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:06.411984 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc31339-f8dc-4ccf-a48a-cef2de88122f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:06.421705 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2dadaa-864f-4f62-b1c0-24454691edfe None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:06.458424 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59c152-c30c-45b3-bb0c-808dc88d1e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:06.459122 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59c152-c30c-45b3-bb0c-808dc88d1e28 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:06.459382 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59c152-c30c-45b3-bb0c-808dc88d1e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:06.459702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da59c152-c30c-45b3-bb0c-808dc88d1e28 None admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:06.460585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da59c152-c30c-45b3-bb0c-808dc88d1e28 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:06.460924 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da59c152-c30c-45b3-bb0c-808dc88d1e28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:06.462543 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da59c152-c30c-45b3-bb0c-808dc88d1e28 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:06.473705 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:06.483578 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 313/644] 10.222.0.60 () {56 vars in 1137 bytes} [Wed Jul 13 23:02:06 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:06.483578 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59c152-c30c-45b3-bb0c-808dc88d1e28 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:06.511933 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:06.533310 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 332/645] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:02:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 23:02:06.534184 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc31339-f8dc-4ccf-a48a-cef2de88122f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:06.598195 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da59c152-c30c-45b3-bb0c-808dc88d1e28 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:06.635586 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e108ea-2817-4040-87bd-53a79c82c1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:06.636255 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e108ea-2817-4040-87bd-53a79c82c1a5 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:06.636534 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e108ea-2817-4040-87bd-53a79c82c1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:06.636809 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e108ea-2817-4040-87bd-53a79c82c1a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:06.637530 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e108ea-2817-4040-87bd-53a79c82c1a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:06.637830 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e108ea-2817-4040-87bd-53a79c82c1a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:06.639195 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e108ea-2817-4040-87bd-53a79c82c1a5 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:06.643597 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc31339-f8dc-4ccf-a48a-cef2de88122f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:06.651425 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 314/646] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:06.652259 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e108ea-2817-4040-87bd-53a79c82c1a5 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:06.684608 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3efaa9-4e7c-4e1e-83af-051a40af254d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:06.685200 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3efaa9-4e7c-4e1e-83af-051a40af254d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:06.685479 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3efaa9-4e7c-4e1e-83af-051a40af254d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:06.685766 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3efaa9-4e7c-4e1e-83af-051a40af254d None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:06.686494 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3efaa9-4e7c-4e1e-83af-051a40af254d None admin] RBAC: Adding query filter params (name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:06.686797 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3efaa9-4e7c-4e1e-83af-051a40af254d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:06.688212 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3efaa9-4e7c-4e1e-83af-051a40af254d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:06.697736 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 333/647] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jul 13 23:02:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:06.698801 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3efaa9-4e7c-4e1e-83af-051a40af254d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:06.717493 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:06.756776 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:06.771851 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e108ea-2817-4040-87bd-53a79c82c1a5 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:06.801271 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3efaa9-4e7c-4e1e-83af-051a40af254d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:06.806402 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c865dc-ddbf-44ce-a476-dbad4abfe61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:06.807043 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c865dc-ddbf-44ce-a476-dbad4abfe61d None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:06.807321 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c865dc-ddbf-44ce-a476-dbad4abfe61d None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:06.807612 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c865dc-ddbf-44ce-a476-dbad4abfe61d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:06.834370 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca60141d-e691-46d7-b4a1-c854b0b0b094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:06.835150 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca60141d-e691-46d7-b4a1-c854b0b0b094 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:06.835432 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca60141d-e691-46d7-b4a1-c854b0b0b094 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:06.835734 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca60141d-e691-46d7-b4a1-c854b0b0b094 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:06.846506 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca60141d-e691-46d7-b4a1-c854b0b0b094 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:06.848034 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca60141d-e691-46d7-b4a1-c854b0b0b094 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:06.865094 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-ca60141d-e691-46d7-b4a1-c854b0b0b094 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:06.865094 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:06.865094 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:06.865094 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:06.865094 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:06.865094 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:06.865094 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:06.865094 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:06.865094 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:06.865094 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:06.865094 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:06.865094 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:06.865094 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:06.865094 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:06.865094 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:06.865973 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:06.865973 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:06.865973 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:06.865973 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:06.865973 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:06.865973 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:06.865973 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:06.865973 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:06.865973 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:06.865973 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:06.865973 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:06.865973 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:06.865973 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:06.865973 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:06.865973 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:06.865973 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:06.865973 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:06.867473 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:06.867473 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:06.867473 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:06.867473 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:06.867473 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 334/648] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:06.867803 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca60141d-e691-46d7-b4a1-c854b0b0b094 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:06.877317 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c865dc-ddbf-44ce-a476-dbad4abfe61d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:06.878778 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c865dc-ddbf-44ce-a476-dbad4abfe61d None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:06.934228 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:06.965136 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:06.965662 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca60141d-e691-46d7-b4a1-c854b0b0b094 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:06.978426 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 315/649] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:02:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 23:02:06.979239 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c865dc-ddbf-44ce-a476-dbad4abfe61d None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:06.999947 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316559b-8b4a-43ca-ad6f-b95fe936b1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:07.000735 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316559b-8b4a-43ca-ad6f-b95fe936b1e8 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:07.001014 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316559b-8b4a-43ca-ad6f-b95fe936b1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:07.001287 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316559b-8b4a-43ca-ad6f-b95fe936b1e8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:07.009025 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2316559b-8b4a-43ca-ad6f-b95fe936b1e8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:07.010475 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2316559b-8b4a-43ca-ad6f-b95fe936b1e8 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:07.024291 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-2316559b-8b4a-43ca-ad6f-b95fe936b1e8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 13 23:02:07.024291 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:07.024291 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:07.024291 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:07.024291 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:07.024291 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:07.024291 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:07.024291 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:07.024291 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:07.024291 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:07.024291 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:07.024291 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:07.024291 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 23:02:07.024291 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 23:02:07.024291 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 23:02:07.024291 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 23:02:07.025411 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:07.025411 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:07.025411 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 23:02:07.025411 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:07.025411 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 23:02:07.025411 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:07.025411 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:07.025411 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:07.025411 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:07.025411 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:07.025411 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:07.025411 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:07.025411 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:07.025411 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:07.025411 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:07.025411 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:07.025411 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:07.025411 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:07.026350 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:07.026350 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:07.026350 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 23:02:07.026350 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 23:02:07.026350 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 23:02:07.026350 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 23:02:07.026350 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 23:02:07.026350 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 23:02:07.026350 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 13 23:02:07.026350 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:07.026350 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 335/650] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:02:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:07.027166 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c3120-5971-4061-bfa7-192a56a28c5f None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:07.027166 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c3120-5971-4061-bfa7-192a56a28c5f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:07.027166 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c3120-5971-4061-bfa7-192a56a28c5f None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:07.028606 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 336/651] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:07.029103 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c3120-5971-4061-bfa7-192a56a28c5f None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:07.081876 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c865dc-ddbf-44ce-a476-dbad4abfe61d None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:07.115545 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2f7a0-5d76-417e-b3ce-56757fc75516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:07.116352 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2f7a0-5d76-417e-b3ce-56757fc75516 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:07.117183 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2f7a0-5d76-417e-b3ce-56757fc75516 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:07.117458 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2f7a0-5d76-417e-b3ce-56757fc75516 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:07.127278 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a2f7a0-5d76-417e-b3ce-56757fc75516 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:07.128765 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a2f7a0-5d76-417e-b3ce-56757fc75516 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:07.130588 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c3120-5971-4061-bfa7-192a56a28c5f None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:07.145632 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-28a2f7a0-5d76-417e-b3ce-56757fc75516 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:07.145632 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:07.145632 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:07.145632 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:07.145632 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:07.145632 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:07.145632 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:07.145632 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:07.145632 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:07.145632 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:07.145632 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:07.145632 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:07.145632 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:07.145632 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:07.145632 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:07.146567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:07.146567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:07.146567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:07.146567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:07.146567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:07.146567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:07.146567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:07.146567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:07.146567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:07.146567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:07.146567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:07.146567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:07.146567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:07.146567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:07.146567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:07.146567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:07.146567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:07.147433 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:07.147433 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:07.147433 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:07.147433 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:07.147433 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 316/652] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:07.148433 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2f7a0-5d76-417e-b3ce-56757fc75516 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:07.162839 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:07.169826 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064c71f-4644-45a5-9945-f564e4df0712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:07.170489 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064c71f-4644-45a5-9945-f564e4df0712 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:07.170785 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064c71f-4644-45a5-9945-f564e4df0712 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:07.171060 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064c71f-4644-45a5-9945-f564e4df0712 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:07.171814 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a064c71f-4644-45a5-9945-f564e4df0712 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:07.172120 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a064c71f-4644-45a5-9945-f564e4df0712 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:07.173483 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a064c71f-4644-45a5-9945-f564e4df0712 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:07.185646 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 337/653] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:07.186285 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064c71f-4644-45a5-9945-f564e4df0712 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:07.191752 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:07.250105 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2f7a0-5d76-417e-b3ce-56757fc75516 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:07.283354 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064c71f-4644-45a5-9945-f564e4df0712 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:07.283961 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0698194a-d08d-44fd-8582-a12383090c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:07.284746 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0698194a-d08d-44fd-8582-a12383090c3d None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:07.285026 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0698194a-d08d-44fd-8582-a12383090c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:07.285290 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0698194a-d08d-44fd-8582-a12383090c3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:07.295483 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0698194a-d08d-44fd-8582-a12383090c3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:07.296970 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0698194a-d08d-44fd-8582-a12383090c3d None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:07.314808 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-0698194a-d08d-44fd-8582-a12383090c3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:07.314808 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:07.314808 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:07.314808 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:07.314808 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:07.314808 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:07.314808 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:07.314808 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:07.314808 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:07.314808 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:07.314808 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:07.314808 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:07.314808 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:07.314808 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:07.314808 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:07.320515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:07.320515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:07.320515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:07.320515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:07.320515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:07.320515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:07.320515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:07.320515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:07.320515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:07.320515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:07.320515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:07.320515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:07.320515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:07.320515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:07.320515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:07.320515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:07.320515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:07.321718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:07.321718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:07.321718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:07.321718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:07.321718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 317/654] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:07.322026 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d42e822-13df-4232-825d-578e022b9b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:07.322026 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42e822-13df-4232-825d-578e022b9b05 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:07.322026 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42e822-13df-4232-825d-578e022b9b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:07.322235 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2885a5a-5a12-40b5-84a9-577be184b73f None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:07.322306 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42e822-13df-4232-825d-578e022b9b05 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:07.322373 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2885a5a-5a12-40b5-84a9-577be184b73f None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:07.322373 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2885a5a-5a12-40b5-84a9-577be184b73f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:07.322493 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d42e822-13df-4232-825d-578e022b9b05 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:07.322493 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d42e822-13df-4232-825d-578e022b9b05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:07.322493 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d42e822-13df-4232-825d-578e022b9b05 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:07.334133 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-b2885a5a-5a12-40b5-84a9-577be184b73f None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:07.335908 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-b2885a5a-5a12-40b5-84a9-577be184b73f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:07.338196 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 338/655] 10.222.0.60 () {56 vars in 1137 bytes} [Wed Jul 13 23:02:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:07.339015 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d42e822-13df-4232-825d-578e022b9b05 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:07.393572 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:07.408712 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 318/656] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:02:07.409583 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2885a5a-5a12-40b5-84a9-577be184b73f None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:07.442887 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:07.444016 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d42e822-13df-4232-825d-578e022b9b05 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:07.479696 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb14f2-2933-4a88-804b-9e0ecc5b1948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:07.480354 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb14f2-2933-4a88-804b-9e0ecc5b1948 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:07.480636 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb14f2-2933-4a88-804b-9e0ecc5b1948 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:07.480932 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb14f2-2933-4a88-804b-9e0ecc5b1948 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:07.481911 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb14f2-2933-4a88-804b-9e0ecc5b1948 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:07.481911 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb14f2-2933-4a88-804b-9e0ecc5b1948 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:07.483331 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb14f2-2933-4a88-804b-9e0ecc5b1948 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:07.494357 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 339/657] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:07.495164 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb14f2-2933-4a88-804b-9e0ecc5b1948 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:07.519483 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2885a5a-5a12-40b5-84a9-577be184b73f None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:07.553429 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2828a6b-74ef-466f-9dc1-438522c17bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:07.554226 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2828a6b-74ef-466f-9dc1-438522c17bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:07.554505 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2828a6b-74ef-466f-9dc1-438522c17bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:07.554896 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2828a6b-74ef-466f-9dc1-438522c17bf0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:07.561866 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2828a6b-74ef-466f-9dc1-438522c17bf0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:07.563329 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2828a6b-74ef-466f-9dc1-438522c17bf0 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:07.578172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-f2828a6b-74ef-466f-9dc1-438522c17bf0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:07.578172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:07.578172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:07.578172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:07.578172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:07.578172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:07.578172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:07.578172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:07.578172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:07.578172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:07.578172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:07.578172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:07.578172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 23:02:07.578172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 23:02:07.578172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 23:02:07.579127 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 23:02:07.579127 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:07.579127 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:07.579127 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:07.579127 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:07.579127 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:07.579127 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:07.579127 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:07.579127 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:07.579127 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:07.579127 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:07.579127 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:07.579127 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:07.579127 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:07.579127 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:07.579127 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:07.579127 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:07.580192 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 23:02:07.580192 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 23:02:07.580192 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 23:02:07.580192 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 23:02:07.580192 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 23:02:07.580192 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 23:02:07.580192 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:07.580192 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:07.580192 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 319/658] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:02:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:07.580192 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2828a6b-74ef-466f-9dc1-438522c17bf0 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:07.604495 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb14f2-2933-4a88-804b-9e0ecc5b1948 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:07.613246 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:07.638220 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962e707-16f9-467b-a5c3-9918b83fbbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:07.638886 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962e707-16f9-467b-a5c3-9918b83fbbf7 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:07.639163 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962e707-16f9-467b-a5c3-9918b83fbbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:07.639436 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962e707-16f9-467b-a5c3-9918b83fbbf7 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:07.640189 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1962e707-16f9-467b-a5c3-9918b83fbbf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:07.640491 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1962e707-16f9-467b-a5c3-9918b83fbbf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:07.641872 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1962e707-16f9-467b-a5c3-9918b83fbbf7 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:07.654906 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 340/659] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:07.655835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962e707-16f9-467b-a5c3-9918b83fbbf7 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:07.664668 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:07.683389 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2828a6b-74ef-466f-9dc1-438522c17bf0 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:07.717331 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa8a1d-fd37-4892-a066-b31675e89330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:07.717973 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa8a1d-fd37-4892-a066-b31675e89330 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:07.718249 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa8a1d-fd37-4892-a066-b31675e89330 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:07.718527 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa8a1d-fd37-4892-a066-b31675e89330 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:07.754133 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962e707-16f9-467b-a5c3-9918b83fbbf7 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:07.783811 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa8a1d-fd37-4892-a066-b31675e89330 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:07.785204 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa8a1d-fd37-4892-a066-b31675e89330 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:07.788007 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646351fa-ff84-4c7f-bfc3-5c6ccca6ddf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:07.788805 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646351fa-ff84-4c7f-bfc3-5c6ccca6ddf9 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:07.789086 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646351fa-ff84-4c7f-bfc3-5c6ccca6ddf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:07.789366 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646351fa-ff84-4c7f-bfc3-5c6ccca6ddf9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:07.799815 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646351fa-ff84-4c7f-bfc3-5c6ccca6ddf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:02:07.807657 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-646351fa-ff84-4c7f-bfc3-5c6ccca6ddf9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:02:07.807657 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:07.807657 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:07.807657 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:07.807657 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:07.807657 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:07.807657 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:07.807657 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:07.807657 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:07.807657 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:07.807657 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:07.807657 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:07.807657 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:02:07.807657 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:02:07.807657 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:02:07.807657 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:02:07.809246 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:02:07.809246 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:02:07.809246 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:02:07.809246 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:02:07.809246 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:07.809246 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:07.809246 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:07.809246 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:07.809246 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:07.809246 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:07.809246 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:07.809246 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:07.809246 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:07.809246 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:07.809246 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:07.809246 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:07.810074 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:07.810074 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:07.810074 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:07.810074 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:07.810074 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:02:07.810074 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:02:07.810074 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:02:07.810074 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:02:07.810074 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:02:07.810074 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:02:07.810074 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:02:07.810074 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:07.810074 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 341/660] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jul 13 23:02:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:07.810074 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646351fa-ff84-4c7f-bfc3-5c6ccca6ddf9 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:07.840015 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:07.872488 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:07.891793 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 320/661] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:02:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 23:02:07.892602 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa8a1d-fd37-4892-a066-b31675e89330 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:07.912507 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646351fa-ff84-4c7f-bfc3-5c6ccca6ddf9 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:07.946855 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471c93cd-cf63-45ca-8630-93ce880703ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:07.947679 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c93cd-cf63-45ca-8630-93ce880703ff None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:07.947961 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c93cd-cf63-45ca-8630-93ce880703ff None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:07.948232 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c93cd-cf63-45ca-8630-93ce880703ff None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:07.955826 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471c93cd-cf63-45ca-8630-93ce880703ff None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:07.957285 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471c93cd-cf63-45ca-8630-93ce880703ff None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:07.971318 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-471c93cd-cf63-45ca-8630-93ce880703ff None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 13 23:02:07.971318 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:07.971318 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:07.971318 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:07.971318 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:07.971318 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:07.971318 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:07.971318 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:07.971318 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:07.971318 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:07.971318 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:07.971318 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:07.971318 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 23:02:07.971318 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 23:02:07.971318 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 23:02:07.971318 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 23:02:07.972420 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:07.972420 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:07.972420 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 23:02:07.972420 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:07.972420 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 23:02:07.972420 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:07.972420 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:07.972420 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:07.972420 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:07.972420 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:07.972420 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:07.972420 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:07.972420 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:07.972420 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:07.972420 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:07.972420 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:07.972420 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:07.972420 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:07.973790 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:07.973790 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:07.973790 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 23:02:07.973790 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 23:02:07.973790 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 23:02:07.973790 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 23:02:07.973790 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 23:02:07.973790 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 23:02:07.973790 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 13 23:02:07.973790 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:07.973790 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 342/662] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jul 13 23:02:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:07.973790 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471c93cd-cf63-45ca-8630-93ce880703ff None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:07.994539 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa8a1d-fd37-4892-a066-b31675e89330 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:08.029210 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6924b6-da96-47e4-9947-6a3cc0b2b0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:08.029871 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6924b6-da96-47e4-9947-6a3cc0b2b0ce None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:08.030149 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6924b6-da96-47e4-9947-6a3cc0b2b0ce None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:08.030426 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6924b6-da96-47e4-9947-6a3cc0b2b0ce None admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:08.031214 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6924b6-da96-47e4-9947-6a3cc0b2b0ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:08.031523 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6924b6-da96-47e4-9947-6a3cc0b2b0ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:08.033004 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6924b6-da96-47e4-9947-6a3cc0b2b0ce None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:08.044526 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 321/663] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jul 13 23:02:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:08.045275 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6924b6-da96-47e4-9947-6a3cc0b2b0ce None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:08.064330 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:08.079157 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471c93cd-cf63-45ca-8630-93ce880703ff None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:08.090930 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:08.112344 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a514317-458d-4904-99b7-4624170779f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:08.113132 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a514317-458d-4904-99b7-4624170779f8 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:08.113416 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a514317-458d-4904-99b7-4624170779f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:08.113687 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a514317-458d-4904-99b7-4624170779f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:08.120762 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a514317-458d-4904-99b7-4624170779f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:08.122203 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a514317-458d-4904-99b7-4624170779f8 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:08.136649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-8a514317-458d-4904-99b7-4624170779f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:08.136649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:08.136649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:08.136649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:08.136649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:08.136649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:08.136649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:08.136649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:08.136649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:08.136649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:08.136649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:08.136649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:08.136649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 23:02:08.136649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 23:02:08.136649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 23:02:08.137765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 23:02:08.137765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:08.137765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:08.137765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:08.137765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:08.137765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:08.137765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:08.137765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:08.137765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:08.137765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:08.137765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:08.137765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:08.137765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:08.137765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:08.137765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:08.137765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:08.137765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:08.138708 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 23:02:08.138708 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 23:02:08.138708 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 23:02:08.138708 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 23:02:08.138708 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 23:02:08.138708 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 23:02:08.138708 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:08.138708 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:08.138708 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 343/664] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:02:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:08.139230 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a514317-458d-4904-99b7-4624170779f8 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:08.153279 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6924b6-da96-47e4-9947-6a3cc0b2b0ce None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:08.185609 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d93b7a-a6f2-4307-ac58-0cdbd7111e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:08.186281 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d93b7a-a6f2-4307-ac58-0cdbd7111e01 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:08.186559 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d93b7a-a6f2-4307-ac58-0cdbd7111e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:08.186826 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d93b7a-a6f2-4307-ac58-0cdbd7111e01 None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:08.187639 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d93b7a-a6f2-4307-ac58-0cdbd7111e01 None admin] RBAC: Adding query filter params (name=service) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:08.187946 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d93b7a-a6f2-4307-ac58-0cdbd7111e01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:08.189365 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d93b7a-a6f2-4307-ac58-0cdbd7111e01 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:08.199327 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 322/665] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jul 13 23:02:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:08.200154 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d93b7a-a6f2-4307-ac58-0cdbd7111e01 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:08.245366 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a514317-458d-4904-99b7-4624170779f8 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:08.278533 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ef9ec-21b5-4a7a-a8f1-a6da3d464aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:08.279384 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ef9ec-21b5-4a7a-a8f1-a6da3d464aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:08.279696 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ef9ec-21b5-4a7a-a8f1-a6da3d464aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:08.279975 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ef9ec-21b5-4a7a-a8f1-a6da3d464aa1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:08.291171 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0ef9ec-21b5-4a7a-a8f1-a6da3d464aa1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:08.292709 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0ef9ec-21b5-4a7a-a8f1-a6da3d464aa1 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:08.300696 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d93b7a-a6f2-4307-ac58-0cdbd7111e01 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:08.302164 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:08.303168 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:08.313197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-ff0ef9ec-21b5-4a7a-a8f1-a6da3d464aa1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:08.313197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:08.313197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:08.313197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:08.313197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:08.313197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:08.313197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:08.313197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:08.313197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:08.313197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:08.313197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:08.313197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:08.313197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:08.313197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:08.313197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:08.316454 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:08.316454 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:08.316454 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:08.316454 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:08.316454 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:08.316454 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:08.316454 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:08.316454 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:08.316454 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:08.316454 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:08.316454 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:08.316454 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:08.316454 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:08.316454 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:08.316454 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:08.316454 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:08.316454 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:08.317327 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:08.317327 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:08.317327 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:08.317327 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:08.317327 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 344/666] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:08.317327 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ef9ec-21b5-4a7a-a8f1-a6da3d464aa1 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:08.341596 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1365ca48-acd5-4e37-ab96-768a120e2050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:08.342280 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365ca48-acd5-4e37-ab96-768a120e2050 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:08.342561 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365ca48-acd5-4e37-ab96-768a120e2050 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:08.342840 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365ca48-acd5-4e37-ab96-768a120e2050 None admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:08.343723 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1365ca48-acd5-4e37-ab96-768a120e2050 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:08.344038 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1365ca48-acd5-4e37-ab96-768a120e2050 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:08.345485 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1365ca48-acd5-4e37-ab96-768a120e2050 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:08.364276 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 323/667] 10.222.0.60 () {56 vars in 1139 bytes} [Wed Jul 13 23:02:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:08.365212 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1365ca48-acd5-4e37-ab96-768a120e2050 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:08.420130 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ef9ec-21b5-4a7a-a8f1-a6da3d464aa1 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:08.453900 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c63a9f-c58e-4ba7-bd98-39c6a8d3d0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:08.454552 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c63a9f-c58e-4ba7-bd98-39c6a8d3d0bb None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:08.454851 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c63a9f-c58e-4ba7-bd98-39c6a8d3d0bb None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:08.455188 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c63a9f-c58e-4ba7-bd98-39c6a8d3d0bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:08.455994 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c63a9f-c58e-4ba7-bd98-39c6a8d3d0bb None admin] RBAC: Adding query filter params (role.id=92417b14d4984ed9956dcd14f97564f3, scope.project.id=ab0a5581e4e444c293815e1304e20c05, user.id=4789384d158b4995bd30e1e37e30f5f1) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:08.456311 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c63a9f-c58e-4ba7-bd98-39c6a8d3d0bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:08.457760 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c63a9f-c58e-4ba7-bd98-39c6a8d3d0bb None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:08.463902 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 345/668] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:02:08 2022] GET /identity/v3/role_assignments?user.id=4789384d158b4995bd30e1e37e30f5f1&scope.project.id=ab0a5581e4e444c293815e1304e20c05&role.id=92417b14d4984ed9956dcd14f97564f3 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:08.464296 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1365ca48-acd5-4e37-ab96-768a120e2050 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:08.464511 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c63a9f-c58e-4ba7-bd98-39c6a8d3d0bb None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:08.510601 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c439304-fcf3-4aae-8b43-06954030dd7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:08.511278 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c439304-fcf3-4aae-8b43-06954030dd7d None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:08.511590 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c439304-fcf3-4aae-8b43-06954030dd7d None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:08.511874 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c439304-fcf3-4aae-8b43-06954030dd7d None admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:08.512700 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c439304-fcf3-4aae-8b43-06954030dd7d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:08.513009 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c439304-fcf3-4aae-8b43-06954030dd7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:08.514519 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c439304-fcf3-4aae-8b43-06954030dd7d None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:08.529176 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 324/669] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jul 13 23:02:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:08.529885 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c439304-fcf3-4aae-8b43-06954030dd7d None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:08.556294 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:08.558612 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:08.609797 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c63a9f-c58e-4ba7-bd98-39c6a8d3d0bb None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:08.646764 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c439304-fcf3-4aae-8b43-06954030dd7d None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:08.655779 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8d754-b4c0-4b3b-a82a-dda52b4dd577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:08.656595 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8d754-b4c0-4b3b-a82a-dda52b4dd577 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:08.656877 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8d754-b4c0-4b3b-a82a-dda52b4dd577 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:08.657150 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8d754-b4c0-4b3b-a82a-dda52b4dd577 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:08.669578 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b8d754-b4c0-4b3b-a82a-dda52b4dd577 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:08.671076 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b8d754-b4c0-4b3b-a82a-dda52b4dd577 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:08.689861 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2adef48-2eac-4d3d-abf6-d6f7270708b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:08.690543 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2adef48-2eac-4d3d-abf6-d6f7270708b1 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:08.690827 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2adef48-2eac-4d3d-abf6-d6f7270708b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:08.691103 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2adef48-2eac-4d3d-abf6-d6f7270708b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:08.691897 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2adef48-2eac-4d3d-abf6-d6f7270708b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:08.692209 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2adef48-2eac-4d3d-abf6-d6f7270708b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:08.692940 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-55b8d754-b4c0-4b3b-a82a-dda52b4dd577 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:08.692940 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:08.692940 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:08.692940 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:08.692940 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:08.692940 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:08.692940 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:08.692940 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:08.692940 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:08.692940 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:08.692940 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:08.692940 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:08.692940 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:08.692940 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:08.692940 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:08.693875 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:08.693875 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:08.693875 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:08.693875 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:08.693875 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:08.693875 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:08.693875 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:08.693875 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:08.693875 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:08.693875 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:08.693875 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:08.693875 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:08.693875 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:08.693875 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:08.693875 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:08.693875 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:08.693875 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:08.694826 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:08.694826 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:08.694826 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:08.694826 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:08.695046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2adef48-2eac-4d3d-abf6-d6f7270708b1 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:08.695126 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 346/670] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:08.695126 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8d754-b4c0-4b3b-a82a-dda52b4dd577 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:08.709033 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 325/671] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:08.709923 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2adef48-2eac-4d3d-abf6-d6f7270708b1 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:08.809830 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:08.810341 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8d754-b4c0-4b3b-a82a-dda52b4dd577 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:08.831028 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:08.831445 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2adef48-2eac-4d3d-abf6-d6f7270708b1 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:08.854688 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7268390-2bb0-4b94-9fc7-e3cb8f075e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:08.855348 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7268390-2bb0-4b94-9fc7-e3cb8f075e1b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:08.855667 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7268390-2bb0-4b94-9fc7-e3cb8f075e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:08.855943 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7268390-2bb0-4b94-9fc7-e3cb8f075e1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:08.870465 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e7e28-a1dd-4a07-b662-da02b9816890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:08.871128 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e7e28-a1dd-4a07-b662-da02b9816890 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:08.871411 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e7e28-a1dd-4a07-b662-da02b9816890 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:08.871721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10e7e28-a1dd-4a07-b662-da02b9816890 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:08.872558 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10e7e28-a1dd-4a07-b662-da02b9816890 None admin] RBAC: Adding query filter params (role.id=92417b14d4984ed9956dcd14f97564f3, scope.project.id=ab0a5581e4e444c293815e1304e20c05, user.id=54661bad914f455cb00d42ec692fde11) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:08.872869 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10e7e28-a1dd-4a07-b662-da02b9816890 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:08.874327 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10e7e28-a1dd-4a07-b662-da02b9816890 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:08.880931 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 326/672] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:02:08 2022] GET /identity/v3/role_assignments?user.id=54661bad914f455cb00d42ec692fde11&scope.project.id=ab0a5581e4e444c293815e1304e20c05&role.id=92417b14d4984ed9956dcd14f97564f3 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:08.888695 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e7e28-a1dd-4a07-b662-da02b9816890 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:08.937791 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7268390-2bb0-4b94-9fc7-e3cb8f075e1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:08.939710 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7268390-2bb0-4b94-9fc7-e3cb8f075e1b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:09.017882 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10e7e28-a1dd-4a07-b662-da02b9816890 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:09.058750 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 347/673] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:02:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 23:02:09.058986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7268390-2bb0-4b94-9fc7-e3cb8f075e1b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:09.063203 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2a46a-a422-471a-823d-7c9b0fe67d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:09.063753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2a46a-a422-471a-823d-7c9b0fe67d70 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:09.064037 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2a46a-a422-471a-823d-7c9b0fe67d70 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:09.064395 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2a46a-a422-471a-823d-7c9b0fe67d70 None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:09.065280 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe2a46a-a422-471a-823d-7c9b0fe67d70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:09.065387 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe2a46a-a422-471a-823d-7c9b0fe67d70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:09.068065 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe2a46a-a422-471a-823d-7c9b0fe67d70 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:09.083097 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 327/674] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:09.088567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2a46a-a422-471a-823d-7c9b0fe67d70 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:09.090351 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:09.099849 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:09.187576 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7268390-2bb0-4b94-9fc7-e3cb8f075e1b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:09.222664 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2a46a-a422-471a-823d-7c9b0fe67d70 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:09.226392 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38fa7eb-57cd-4d47-9489-ec65d8fca1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:09.227460 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38fa7eb-57cd-4d47-9489-ec65d8fca1cb None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:09.228103 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38fa7eb-57cd-4d47-9489-ec65d8fca1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:09.228515 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38fa7eb-57cd-4d47-9489-ec65d8fca1cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:09.235872 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38fa7eb-57cd-4d47-9489-ec65d8fca1cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:09.237756 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38fa7eb-57cd-4d47-9489-ec65d8fca1cb None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:09.252673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-c38fa7eb-57cd-4d47-9489-ec65d8fca1cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:09.252673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:09.252673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:09.252673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:09.252673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:09.252673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:09.252673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:09.252673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:09.252673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:09.252673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:09.252673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:09.252673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:09.252673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 23:02:09.252673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 23:02:09.252673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 23:02:09.253691 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 23:02:09.253691 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:09.253691 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:09.253691 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:09.253691 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:09.253691 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:09.253691 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:09.253691 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:09.253691 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:09.253691 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:09.253691 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:09.253691 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:09.253691 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:09.253691 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:09.253691 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:09.253691 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:09.253691 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:09.254551 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 23:02:09.254551 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 23:02:09.254551 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 23:02:09.254551 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 23:02:09.254551 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 23:02:09.254551 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 23:02:09.254551 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:09.254551 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:09.257819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 348/675] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:02:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:09.258978 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38fa7eb-57cd-4d47-9489-ec65d8fca1cb None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:09.267559 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c6a3b3-0c0c-4089-bb1b-2c0c5a232be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:09.267559 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c6a3b3-0c0c-4089-bb1b-2c0c5a232be2 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:09.267559 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c6a3b3-0c0c-4089-bb1b-2c0c5a232be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:09.267559 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c6a3b3-0c0c-4089-bb1b-2c0c5a232be2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:09.274254 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c6a3b3-0c0c-4089-bb1b-2c0c5a232be2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:09.276198 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c6a3b3-0c0c-4089-bb1b-2c0c5a232be2 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:09.299640 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-c2c6a3b3-0c0c-4089-bb1b-2c0c5a232be2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:09.299640 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:09.299640 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:09.299640 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:09.299640 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:09.299640 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:09.299640 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:09.299640 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:09.299640 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:09.299640 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:09.299640 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:09.299640 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:09.299640 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:09.299640 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:09.299640 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:09.300736 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:09.300736 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:09.300736 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:09.300736 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:09.300736 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:09.300736 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:09.300736 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:09.300736 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:09.300736 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:09.300736 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:09.300736 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:09.300736 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:09.300736 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:09.300736 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:09.300736 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:09.300736 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:09.300736 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:09.301807 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:09.301807 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:09.301807 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:09.301807 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:09.303618 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 328/676] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:09.304610 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c6a3b3-0c0c-4089-bb1b-2c0c5a232be2 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:09.364052 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:09.374501 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:09.397924 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38fa7eb-57cd-4d47-9489-ec65d8fca1cb None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:09.420014 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c6a3b3-0c0c-4089-bb1b-2c0c5a232be2 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:09.441454 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6814237a-6dfb-4f48-92f4-ec1baf889bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:09.442255 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6814237a-6dfb-4f48-92f4-ec1baf889bdf None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:09.442537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6814237a-6dfb-4f48-92f4-ec1baf889bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:09.442813 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6814237a-6dfb-4f48-92f4-ec1baf889bdf None admin] PATH_INFO: `/v3/users/swift` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:09.451796 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6814237a-6dfb-4f48-92f4-ec1baf889bdf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:09.453258 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6814237a-6dfb-4f48-92f4-ec1baf889bdf None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:09.457646 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1d431-a818-4783-917a-7b0c5ff966df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:09.458564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1d431-a818-4783-917a-7b0c5ff966df None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:09.459047 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1d431-a818-4783-917a-7b0c5ff966df None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:09.459520 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1d431-a818-4783-917a-7b0c5ff966df None admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:09.460581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e1d431-a818-4783-917a-7b0c5ff966df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:09.461106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e1d431-a818-4783-917a-7b0c5ff966df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:09.462738 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e1d431-a818-4783-917a-7b0c5ff966df None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:09.467821 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-6814237a-6dfb-4f48-92f4-ec1baf889bdf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 13 23:02:09.467821 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:09.467821 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:09.467821 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:09.467821 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:09.467821 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:09.467821 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:09.467821 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:09.467821 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:09.467821 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:09.467821 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:09.467821 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:09.467821 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 23:02:09.467821 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 23:02:09.467821 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 23:02:09.467821 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 23:02:09.468742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:09.468742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:09.468742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 23:02:09.468742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:09.468742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 23:02:09.468742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:09.468742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:09.468742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:09.468742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:09.468742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:09.468742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:09.468742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:09.468742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:09.468742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:09.468742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:09.468742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:09.468742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:09.468742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:09.469657 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:09.469657 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:09.469657 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 23:02:09.469657 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 23:02:09.469657 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 23:02:09.469657 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 23:02:09.469657 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 23:02:09.469657 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 23:02:09.469657 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 13 23:02:09.469657 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:09.469657 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 349/677] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jul 13 23:02:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:09.470262 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6814237a-6dfb-4f48-92f4-ec1baf889bdf None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:09.483924 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 329/678] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jul 13 23:02:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:09.485192 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1d431-a818-4783-917a-7b0c5ff966df None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:09.579142 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6814237a-6dfb-4f48-92f4-ec1baf889bdf None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:09.605522 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1d431-a818-4783-917a-7b0c5ff966df None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:09.620459 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f4da90-f416-43a2-8e3c-6fadbb097fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:09.621165 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4da90-f416-43a2-8e3c-6fadbb097fe8 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:09.621783 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4da90-f416-43a2-8e3c-6fadbb097fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:09.622136 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4da90-f416-43a2-8e3c-6fadbb097fe8 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:09.622953 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f4da90-f416-43a2-8e3c-6fadbb097fe8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:09.623336 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f4da90-f416-43a2-8e3c-6fadbb097fe8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:09.625414 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f4da90-f416-43a2-8e3c-6fadbb097fe8 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:09.629370 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:09.644083 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 350/679] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:09.647587 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:09.649663 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f4da90-f416-43a2-8e3c-6fadbb097fe8 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:09.654529 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8d27d-d629-4fd8-a05b-2df644765e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:09.655474 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8d27d-d629-4fd8-a05b-2df644765e55 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:09.655955 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8d27d-d629-4fd8-a05b-2df644765e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:09.656439 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8d27d-d629-4fd8-a05b-2df644765e55 None admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:09.657463 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba8d27d-d629-4fd8-a05b-2df644765e55 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:09.657969 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba8d27d-d629-4fd8-a05b-2df644765e55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:09.659618 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba8d27d-d629-4fd8-a05b-2df644765e55 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:09.680238 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 330/680] 10.222.0.60 () {56 vars in 1135 bytes} [Wed Jul 13 23:02:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:09.681488 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8d27d-d629-4fd8-a05b-2df644765e55 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:09.769443 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f4da90-f416-43a2-8e3c-6fadbb097fe8 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:09.791658 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8d27d-d629-4fd8-a05b-2df644765e55 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:09.806714 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2287bd60-2321-44b4-8de6-70edcf34329c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:09.807771 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287bd60-2321-44b4-8de6-70edcf34329c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:09.807771 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287bd60-2321-44b4-8de6-70edcf34329c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:09.808132 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287bd60-2321-44b4-8de6-70edcf34329c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:09.809048 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2287bd60-2321-44b4-8de6-70edcf34329c None admin] RBAC: Adding query filter params (role.id=92417b14d4984ed9956dcd14f97564f3, scope.project.id=ab0a5581e4e444c293815e1304e20c05, user.id=2ce9e6497fdb4068b23aa3f7b2c19536) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:09.809428 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2287bd60-2321-44b4-8de6-70edcf34329c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:09.810909 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2287bd60-2321-44b4-8de6-70edcf34329c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:09.817612 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 351/681] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:02:09 2022] GET /identity/v3/role_assignments?user.id=2ce9e6497fdb4068b23aa3f7b2c19536&scope.project.id=ab0a5581e4e444c293815e1304e20c05&role.id=92417b14d4984ed9956dcd14f97564f3 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:09.819239 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2287bd60-2321-44b4-8de6-70edcf34329c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:09.831992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91232bd-a5b7-405a-adda-19130ad483d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:09.833092 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91232bd-a5b7-405a-adda-19130ad483d6 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:09.833547 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91232bd-a5b7-405a-adda-19130ad483d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:09.833989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91232bd-a5b7-405a-adda-19130ad483d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:09.841324 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91232bd-a5b7-405a-adda-19130ad483d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:09.843017 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91232bd-a5b7-405a-adda-19130ad483d6 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:09.858850 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-c91232bd-a5b7-405a-adda-19130ad483d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:09.858850 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:09.858850 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:09.858850 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:09.858850 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:09.858850 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:09.858850 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:09.858850 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:09.858850 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:09.858850 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:09.858850 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:09.858850 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:09.858850 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 23:02:09.858850 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 23:02:09.858850 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 23:02:09.859917 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 23:02:09.859917 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:09.859917 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:09.859917 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:09.859917 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:09.859917 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:09.859917 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:09.859917 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:09.859917 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:09.859917 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:09.859917 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:09.859917 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:09.859917 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:09.859917 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:09.859917 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:09.859917 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:09.859917 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:09.860784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 23:02:09.860784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 23:02:09.860784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 23:02:09.860784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 23:02:09.860784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 23:02:09.860784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 23:02:09.860784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:09.860784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:09.863356 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 331/682] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:02:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:09.874156 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91232bd-a5b7-405a-adda-19130ad483d6 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:09.899930 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:09.943566 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2287bd60-2321-44b4-8de6-70edcf34329c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:09.948522 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:09.976701 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8868605-6816-4d53-bf22-db09666fa8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:09.977497 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8868605-6816-4d53-bf22-db09666fa8b5 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:09.977760 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8868605-6816-4d53-bf22-db09666fa8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:09.978038 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8868605-6816-4d53-bf22-db09666fa8b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:10.072836 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8868605-6816-4d53-bf22-db09666fa8b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:10.074376 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8868605-6816-4d53-bf22-db09666fa8b5 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:10.081661 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91232bd-a5b7-405a-adda-19130ad483d6 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:10.140232 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb1d1f9-7d5e-4ca6-8c71-48d58797f8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:10.141254 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb1d1f9-7d5e-4ca6-8c71-48d58797f8fd None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:10.141757 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb1d1f9-7d5e-4ca6-8c71-48d58797f8fd None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:10.142227 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb1d1f9-7d5e-4ca6-8c71-48d58797f8fd None admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:10.143242 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb1d1f9-7d5e-4ca6-8c71-48d58797f8fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:10.143783 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb1d1f9-7d5e-4ca6-8c71-48d58797f8fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:10.145430 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb1d1f9-7d5e-4ca6-8c71-48d58797f8fd None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:10.160066 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 332/683] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jul 13 23:02:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:10.168093 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb1d1f9-7d5e-4ca6-8c71-48d58797f8fd None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:10.171787 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:10.224369 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 352/684] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:02:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 23:02:10.230033 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8868605-6816-4d53-bf22-db09666fa8b5 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:10.280624 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:10.287750 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb1d1f9-7d5e-4ca6-8c71-48d58797f8fd None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:10.339235 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec326a-cb51-4be7-8451-07404885e38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:10.339928 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec326a-cb51-4be7-8451-07404885e38c None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:10.340211 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec326a-cb51-4be7-8451-07404885e38c None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:10.340485 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec326a-cb51-4be7-8451-07404885e38c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:10.341397 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dec326a-cb51-4be7-8451-07404885e38c None admin] RBAC: Adding query filter params (role.id=92417b14d4984ed9956dcd14f97564f3, scope.project.id=ab0a5581e4e444c293815e1304e20c05, user.id=2d670df2aff5478ebf61f55358d554de) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:10.341710 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dec326a-cb51-4be7-8451-07404885e38c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:10.343834 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dec326a-cb51-4be7-8451-07404885e38c None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:10.352970 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 333/685] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:02:10 2022] GET /identity/v3/role_assignments?user.id=2d670df2aff5478ebf61f55358d554de&scope.project.id=ab0a5581e4e444c293815e1304e20c05&role.id=92417b14d4984ed9956dcd14f97564f3 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:02:10.380925 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8868605-6816-4d53-bf22-db09666fa8b5 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:10.425600 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c420c573-226a-40a7-ae3e-85853429a155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:10.426328 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420c573-226a-40a7-ae3e-85853429a155 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:10.426606 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420c573-226a-40a7-ae3e-85853429a155 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:10.426926 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420c573-226a-40a7-ae3e-85853429a155 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:10.439497 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c420c573-226a-40a7-ae3e-85853429a155 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:10.441038 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c420c573-226a-40a7-ae3e-85853429a155 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:10.462090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-c420c573-226a-40a7-ae3e-85853429a155 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:10.462090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:10.462090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:10.462090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:10.462090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:10.462090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:10.462090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:10.462090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:10.462090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:10.462090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:10.462090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:10.462090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:10.462090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:10.462090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:10.462090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:10.463062 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:10.463062 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:10.463062 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:10.463062 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:10.463062 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:10.463062 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:10.463062 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:10.463062 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:10.463062 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:10.463062 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:10.463062 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:10.463062 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:10.463062 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:10.463062 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:10.463062 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:10.463062 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:10.463062 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:10.463983 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:10.463983 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:10.463983 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:10.463983 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:10.468025 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 353/686] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:10.469582 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec326a-cb51-4be7-8451-07404885e38c None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:10.594912 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec326a-cb51-4be7-8451-07404885e38c None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:10.638296 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efdd7c6-9e30-4483-aa58-1752844cb9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:10.639019 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efdd7c6-9e30-4483-aa58-1752844cb9f7 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:10.639305 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efdd7c6-9e30-4483-aa58-1752844cb9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:10.639603 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efdd7c6-9e30-4483-aa58-1752844cb9f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:10.640348 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efdd7c6-9e30-4483-aa58-1752844cb9f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:10.640653 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efdd7c6-9e30-4483-aa58-1752844cb9f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:10.642047 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efdd7c6-9e30-4483-aa58-1752844cb9f7 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:10.655185 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 334/687] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:10.660394 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c420c573-226a-40a7-ae3e-85853429a155 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:10.677382 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:10.716372 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45fb5d-9734-4500-8848-076ba892a561 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:10.716687 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45fb5d-9734-4500-8848-076ba892a561 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:10.716920 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45fb5d-9734-4500-8848-076ba892a561 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:10.718313 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 335/688] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:02:10.723952 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4eac93-dd48-496a-9802-24b63454e794 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:10.724632 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4eac93-dd48-496a-9802-24b63454e794 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:10.725220 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4eac93-dd48-496a-9802-24b63454e794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:10.744467 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-3b4eac93-dd48-496a-9802-24b63454e794 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:10.746428 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-3b4eac93-dd48-496a-9802-24b63454e794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:10.762897 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:10.796701 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c420c573-226a-40a7-ae3e-85853429a155 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:10.835660 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 336/689] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:02:10.841861 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19c644-1aa7-469e-9c42-87cfefacbdfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:10.842670 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19c644-1aa7-469e-9c42-87cfefacbdfc None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:10.842955 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19c644-1aa7-469e-9c42-87cfefacbdfc None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:10.843235 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19c644-1aa7-469e-9c42-87cfefacbdfc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:10.851936 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af19c644-1aa7-469e-9c42-87cfefacbdfc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:10.853479 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af19c644-1aa7-469e-9c42-87cfefacbdfc None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:10.857157 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4eac93-dd48-496a-9802-24b63454e794 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:10.873381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-af19c644-1aa7-469e-9c42-87cfefacbdfc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:10.873381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:10.873381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:10.873381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:10.873381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:10.873381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:10.873381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:10.873381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:10.873381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:10.873381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:10.873381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:10.873381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:10.873381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 23:02:10.873381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 23:02:10.873381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 23:02:10.874353 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 23:02:10.874353 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:10.874353 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:10.874353 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:10.874353 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:10.874353 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:10.874353 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:10.874353 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:10.874353 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:10.874353 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:10.874353 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:10.874353 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:10.874353 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:10.874353 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:10.874353 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:10.874353 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:10.874353 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:10.876012 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 23:02:10.876012 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 23:02:10.876012 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 23:02:10.876012 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 23:02:10.876012 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 23:02:10.876012 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 23:02:10.876012 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:10.876012 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:10.876012 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 354/690] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:02:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:02:10.884639 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19c644-1aa7-469e-9c42-87cfefacbdfc None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:10.985489 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4eac93-dd48-496a-9802-24b63454e794 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:10.994486 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:11.024920 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a086df-8abf-4f9c-ad6a-6b9ae31f9625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:11.026303 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a086df-8abf-4f9c-ad6a-6b9ae31f9625 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:11.026831 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a086df-8abf-4f9c-ad6a-6b9ae31f9625 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:11.026960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19c644-1aa7-469e-9c42-87cfefacbdfc None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:11.027424 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a086df-8abf-4f9c-ad6a-6b9ae31f9625 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:11.039913 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a086df-8abf-4f9c-ad6a-6b9ae31f9625 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:02:11.048267 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-e6a086df-8abf-4f9c-ad6a-6b9ae31f9625 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:02:11.048267 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:11.048267 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:11.048267 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:11.048267 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:11.048267 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:11.048267 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:11.048267 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:11.048267 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:11.048267 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:11.048267 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:11.048267 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:11.048267 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:02:11.048267 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:02:11.048267 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:02:11.048267 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:02:11.049231 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:02:11.049231 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:02:11.049231 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:02:11.049231 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:02:11.049231 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:11.049231 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:11.049231 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:11.049231 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:11.049231 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:11.049231 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:11.049231 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:11.049231 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:11.049231 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:11.049231 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:11.049231 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:11.049231 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:11.050095 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:11.050095 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:11.050095 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:11.050095 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:11.050095 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:02:11.050095 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:02:11.050095 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:02:11.050095 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:02:11.050095 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:02:11.050095 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:02:11.050095 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:02:11.050095 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:11.057735 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 337/691] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jul 13 23:02:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:11.059414 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a086df-8abf-4f9c-ad6a-6b9ae31f9625 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:11.064488 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe9d07-78aa-4a22-8f73-7eec3e5db9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:11.064686 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe9d07-78aa-4a22-8f73-7eec3e5db9b7 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:11.064977 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe9d07-78aa-4a22-8f73-7eec3e5db9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:11.065258 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe9d07-78aa-4a22-8f73-7eec3e5db9b7 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:11.066124 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe9d07-78aa-4a22-8f73-7eec3e5db9b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:11.066457 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe9d07-78aa-4a22-8f73-7eec3e5db9b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:11.068076 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe9d07-78aa-4a22-8f73-7eec3e5db9b7 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:11.076026 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:11.082780 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 355/692] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jul 13 23:02:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:11.092636 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe9d07-78aa-4a22-8f73-7eec3e5db9b7 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:11.214175 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a086df-8abf-4f9c-ad6a-6b9ae31f9625 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:11.263824 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa2b36a-8678-4e1c-8d34-4b2f9cd7bed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:11.264858 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2b36a-8678-4e1c-8d34-4b2f9cd7bed0 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:11.265353 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2b36a-8678-4e1c-8d34-4b2f9cd7bed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:11.265823 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2b36a-8678-4e1c-8d34-4b2f9cd7bed0 None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:11.266776 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa2b36a-8678-4e1c-8d34-4b2f9cd7bed0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:11.267286 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa2b36a-8678-4e1c-8d34-4b2f9cd7bed0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:11.268921 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa2b36a-8678-4e1c-8d34-4b2f9cd7bed0 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:11.274015 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe9d07-78aa-4a22-8f73-7eec3e5db9b7 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:11.286712 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 338/693] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:02:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:02:11.288550 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0867017-2a97-44e3-8163-10a77fc54e23 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:11.289068 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0867017-2a97-44e3-8163-10a77fc54e23 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:11.289564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0867017-2a97-44e3-8163-10a77fc54e23 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:11.295553 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 339/694] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:11.296796 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0867017-2a97-44e3-8163-10a77fc54e23 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:11.310035 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86907e2-9d8e-4c0c-bc9f-60a0c00c754e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:11.310855 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86907e2-9d8e-4c0c-bc9f-60a0c00c754e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:11.311078 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86907e2-9d8e-4c0c-bc9f-60a0c00c754e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:11.311359 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86907e2-9d8e-4c0c-bc9f-60a0c00c754e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:11.312318 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86907e2-9d8e-4c0c-bc9f-60a0c00c754e None admin] RBAC: Adding query filter params (role.id=92417b14d4984ed9956dcd14f97564f3, scope.project.id=ab0a5581e4e444c293815e1304e20c05, user.id=786fc44611804d3ca6280ffcab415569) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:11.312560 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86907e2-9d8e-4c0c-bc9f-60a0c00c754e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:11.314192 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86907e2-9d8e-4c0c-bc9f-60a0c00c754e None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:11.322140 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 356/695] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:02:11 2022] GET /identity/v3/role_assignments?user.id=786fc44611804d3ca6280ffcab415569&scope.project.id=ab0a5581e4e444c293815e1304e20c05&role.id=92417b14d4984ed9956dcd14f97564f3 => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:11.323867 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff4996-f730-4903-8957-93e92492befd None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:11.323983 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff4996-f730-4903-8957-93e92492befd None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:11.324213 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff4996-f730-4903-8957-93e92492befd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:11.340015 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:11.343092 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c5ff4996-f730-4903-8957-93e92492befd None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:11.344363 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c5ff4996-f730-4903-8957-93e92492befd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:11.399937 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:11.426969 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0867017-2a97-44e3-8163-10a77fc54e23 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:11.476265 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 357/696] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:02:11.502464 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff4996-f730-4903-8957-93e92492befd None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:11.511035 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46861f-3bff-416a-a8d2-b7128e823045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:11.511907 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46861f-3bff-416a-a8d2-b7128e823045 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:11.512188 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46861f-3bff-416a-a8d2-b7128e823045 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:11.512466 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46861f-3bff-416a-a8d2-b7128e823045 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:11.527625 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b46861f-3bff-416a-a8d2-b7128e823045 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:11.529075 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b46861f-3bff-416a-a8d2-b7128e823045 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:11.552811 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-4b46861f-3bff-416a-a8d2-b7128e823045 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:11.552811 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:11.552811 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:11.552811 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:11.552811 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:11.552811 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:11.552811 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:11.552811 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:11.552811 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:11.552811 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:11.552811 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:11.552811 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:11.552811 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:11.552811 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:11.552811 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:11.553777 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:11.553777 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:11.553777 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:11.553777 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:11.553777 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:11.553777 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:11.553777 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:11.553777 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:11.553777 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:11.553777 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:11.553777 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:11.553777 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:11.553777 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:11.553777 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:11.553777 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:11.553777 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:11.553777 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:11.555936 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:11.555936 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:11.555936 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:11.555936 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:11.558480 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 340/697] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:11.559684 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46861f-3bff-416a-a8d2-b7128e823045 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:11.648853 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff4996-f730-4903-8957-93e92492befd None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:11.687733 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6d0d68-6fc0-4285-8ea7-86a9679e79da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:11.688599 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d0d68-6fc0-4285-8ea7-86a9679e79da None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:11.688883 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d0d68-6fc0-4285-8ea7-86a9679e79da None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:11.689160 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d0d68-6fc0-4285-8ea7-86a9679e79da None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:11.689856 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6d0d68-6fc0-4285-8ea7-86a9679e79da None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:11.691274 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6d0d68-6fc0-4285-8ea7-86a9679e79da None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:11.692359 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46861f-3bff-416a-a8d2-b7128e823045 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:11.709258 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-ea6d0d68-6fc0-4285-8ea7-86a9679e79da None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 23:02:11.709258 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:11.709258 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 23:02:11.709258 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 23:02:11.709258 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 23:02:11.709258 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 23:02:11.709258 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 23:02:11.709258 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:11.709258 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 23:02:11.709258 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:11.709258 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 23:02:11.709258 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:11.709258 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:11.709258 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:11.709258 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:11.709258 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:11.710897 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:11.710897 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:11.710897 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:11.710897 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:11.710897 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:11.710897 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:11.710897 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:11.710897 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 23:02:11.710897 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 23:02:11.710897 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 23:02:11.710897 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 23:02:11.710897 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:11.710897 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:11.710897 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:11.710897 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:11.710897 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:11.711756 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:11.711756 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:11.711756 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:11.711756 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:11.711756 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:11.711756 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:11.711756 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:11.711756 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:11.711756 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:11.711756 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:11.711756 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:11.711756 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 23:02:11.711756 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:11.711756 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 23:02:11.711756 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:11.711756 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:11.716543 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 358/698] 10.222.0.60 () {56 vars in 1129 bytes} [Wed Jul 13 23:02:11 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:02:11.720294 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6d0d68-6fc0-4285-8ea7-86a9679e79da None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:11.733130 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:11.747318 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25cf481-d7db-430e-a9bd-461fa00ebf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:11.748389 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cf481-d7db-430e-a9bd-461fa00ebf1a None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:11.748860 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cf481-d7db-430e-a9bd-461fa00ebf1a None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:11.749335 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cf481-d7db-430e-a9bd-461fa00ebf1a None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:11.750253 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25cf481-d7db-430e-a9bd-461fa00ebf1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:11.750714 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25cf481-d7db-430e-a9bd-461fa00ebf1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:11.753535 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25cf481-d7db-430e-a9bd-461fa00ebf1a None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:11.767203 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 341/699] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:11.773415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25cf481-d7db-430e-a9bd-461fa00ebf1a None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:11.854475 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6d0d68-6fc0-4285-8ea7-86a9679e79da None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:11.884050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25cf481-d7db-430e-a9bd-461fa00ebf1a None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:11.901464 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998bc0e-dcf5-4621-ba4a-d7aca6f38b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:11.902621 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998bc0e-dcf5-4621-ba4a-d7aca6f38b7b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:11.904319 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998bc0e-dcf5-4621-ba4a-d7aca6f38b7b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:11.904815 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998bc0e-dcf5-4621-ba4a-d7aca6f38b7b None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:11.905805 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3998bc0e-dcf5-4621-ba4a-d7aca6f38b7b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:11.907124 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3998bc0e-dcf5-4621-ba4a-d7aca6f38b7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:11.909420 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3998bc0e-dcf5-4621-ba4a-d7aca6f38b7b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:11.919765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 359/700] 10.222.0.60 () {56 vars in 1124 bytes} [Wed Jul 13 23:02:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:11.926983 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998bc0e-dcf5-4621-ba4a-d7aca6f38b7b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:11.947351 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c0f39-9271-46ed-be13-7b83cf0966e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:11.949645 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c0f39-9271-46ed-be13-7b83cf0966e1 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:11.950325 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c0f39-9271-46ed-be13-7b83cf0966e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:11.951477 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c0f39-9271-46ed-be13-7b83cf0966e1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:11.971754 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408c0f39-9271-46ed-be13-7b83cf0966e1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:11.973765 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408c0f39-9271-46ed-be13-7b83cf0966e1 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:12.000408 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-408c0f39-9271-46ed-be13-7b83cf0966e1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 23:02:12.000408 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:12.000408 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:12.000408 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:12.000408 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:12.000408 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:12.000408 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:12.000408 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:12.000408 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:12.000408 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:12.000408 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:12.000408 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:12.000408 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 23:02:12.000408 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 23:02:12.000408 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 23:02:12.000408 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 23:02:12.001620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:12.001620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:12.001620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 23:02:12.001620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:12.001620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 23:02:12.001620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:12.001620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:12.001620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:12.001620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:12.001620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:12.001620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:12.001620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:12.001620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:12.001620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:12.001620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:12.001620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:12.001620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:12.001620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:12.002849 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:12.002849 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:12.002849 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 23:02:12.002849 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 23:02:12.002849 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 23:02:12.002849 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 23:02:12.002849 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 23:02:12.002849 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 23:02:12.002849 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 23:02:12.002849 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:12.005203 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 342/701] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Jul 13 23:02:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:02:12.013198 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c0f39-9271-46ed-be13-7b83cf0966e1 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:12.052301 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:12.114497 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:12.139252 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998bc0e-dcf5-4621-ba4a-d7aca6f38b7b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:12.182697 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b88e6-7049-4f7f-8dbb-e8f41ceec2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:12.183371 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b88e6-7049-4f7f-8dbb-e8f41ceec2e3 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:12.183672 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b88e6-7049-4f7f-8dbb-e8f41ceec2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:12.183942 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b88e6-7049-4f7f-8dbb-e8f41ceec2e3 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:12.184654 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54b88e6-7049-4f7f-8dbb-e8f41ceec2e3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:12.184955 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54b88e6-7049-4f7f-8dbb-e8f41ceec2e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:12.186377 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54b88e6-7049-4f7f-8dbb-e8f41ceec2e3 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:12.201043 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 360/702] 10.222.0.60 () {56 vars in 1124 bytes} [Wed Jul 13 23:02:11 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:02:12.211465 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c0f39-9271-46ed-be13-7b83cf0966e1 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:12.274491 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13154878-1c6c-4ec9-8ad3-fc669b0f3175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:12.275150 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13154878-1c6c-4ec9-8ad3-fc669b0f3175 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:12.275467 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13154878-1c6c-4ec9-8ad3-fc669b0f3175 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:12.275717 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13154878-1c6c-4ec9-8ad3-fc669b0f3175 None admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:12.276575 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13154878-1c6c-4ec9-8ad3-fc669b0f3175 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:12.276880 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13154878-1c6c-4ec9-8ad3-fc669b0f3175 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:12.278460 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13154878-1c6c-4ec9-8ad3-fc669b0f3175 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:12.297187 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 343/703] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Jul 13 23:02:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:02:12.304735 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b88e6-7049-4f7f-8dbb-e8f41ceec2e3 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:12.457835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b88e6-7049-4f7f-8dbb-e8f41ceec2e3 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:12.483677 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:12.508309 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9db89-4966-4c86-8cf1-04b903e7deb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:12.508978 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9db89-4966-4c86-8cf1-04b903e7deb4 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:12.509257 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9db89-4966-4c86-8cf1-04b903e7deb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:12.509525 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9db89-4966-4c86-8cf1-04b903e7deb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:12.599189 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df9db89-4966-4c86-8cf1-04b903e7deb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:12.600717 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df9db89-4966-4c86-8cf1-04b903e7deb4 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:12.732241 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 361/704] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:02:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 23:02:12.740669 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13154878-1c6c-4ec9-8ad3-fc669b0f3175 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:12.861811 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13154878-1c6c-4ec9-8ad3-fc669b0f3175 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:12.863959 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:12.905863 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d0f4a-7a8e-4861-921d-5086e232b794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:12.906670 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d0f4a-7a8e-4861-921d-5086e232b794 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:12.907267 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d0f4a-7a8e-4861-921d-5086e232b794 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:12.907579 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d0f4a-7a8e-4861-921d-5086e232b794 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:12.919203 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671d0f4a-7a8e-4861-921d-5086e232b794 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:12.920711 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671d0f4a-7a8e-4861-921d-5086e232b794 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:12.939361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-671d0f4a-7a8e-4861-921d-5086e232b794 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:12.939361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:12.939361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:12.939361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:12.939361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:12.939361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:12.939361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:12.939361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:12.939361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:12.939361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:12.939361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:12.939361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:12.939361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:12.939361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:12.939361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:12.940389 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:12.940389 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:12.940389 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:12.940389 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:12.940389 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:12.940389 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:12.940389 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:12.940389 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:12.940389 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:12.940389 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:12.940389 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:12.940389 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:12.940389 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:12.940389 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:12.940389 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:12.940389 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:12.940389 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:12.941292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:12.941292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:12.941292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:12.941292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:12.941292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 344/705] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:12.947342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9db89-4966-4c86-8cf1-04b903e7deb4 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:12.967781 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:13.087219 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9db89-4966-4c86-8cf1-04b903e7deb4 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:13.131779 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70163843-70a3-49f1-950b-14db3167b0d3 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:13.132741 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70163843-70a3-49f1-950b-14db3167b0d3 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:13.133032 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70163843-70a3-49f1-950b-14db3167b0d3 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:13.134677 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 345/706] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:13.140171 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea6db1-5289-4b3a-a4a1-48499ecd26e5 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:13.140459 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea6db1-5289-4b3a-a4a1-48499ecd26e5 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:13.140837 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dea6db1-5289-4b3a-a4a1-48499ecd26e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:13.141388 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2f785d-0d0a-4db2-aa29-2039dd96752f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:13.142070 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f785d-0d0a-4db2-aa29-2039dd96752f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:13.142391 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f785d-0d0a-4db2-aa29-2039dd96752f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:13.142681 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f785d-0d0a-4db2-aa29-2039dd96752f None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:13.143651 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2f785d-0d0a-4db2-aa29-2039dd96752f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:13.143966 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2f785d-0d0a-4db2-aa29-2039dd96752f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:13.145485 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2f785d-0d0a-4db2-aa29-2039dd96752f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:13.159845 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-0dea6db1-5289-4b3a-a4a1-48499ecd26e5 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:13.161150 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-0dea6db1-5289-4b3a-a4a1-48499ecd26e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:13.163593 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 362/707] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:13.164974 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce92f1-0e57-4ef1-85a8-0b384d267687 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:13.165263 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce92f1-0e57-4ef1-85a8-0b384d267687 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:13.165544 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce92f1-0e57-4ef1-85a8-0b384d267687 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:13.167291 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 363/708] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:13.172047 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce92f1-0e57-4ef1-85a8-0b384d267687 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:13.259627 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 346/709] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:02:13.261072 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c7c3d-121a-4612-bf0b-5ceb37419b73 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:13.261469 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c7c3d-121a-4612-bf0b-5ceb37419b73 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:13.262019 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c7c3d-121a-4612-bf0b-5ceb37419b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:13.300476 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-076c7c3d-121a-4612-bf0b-5ceb37419b73 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:13.301792 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-076c7c3d-121a-4612-bf0b-5ceb37419b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:13.308002 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:13.365441 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce92f1-0e57-4ef1-85a8-0b384d267687 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:13.394977 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 347/710] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:02:13.396301 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c7c3d-121a-4612-bf0b-5ceb37419b73 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:13.404963 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737a0e06-5e63-4eea-9ce8-c7930a8ae51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:13.406031 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a0e06-5e63-4eea-9ce8-c7930a8ae51b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:13.406145 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a0e06-5e63-4eea-9ce8-c7930a8ae51b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:13.406414 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a0e06-5e63-4eea-9ce8-c7930a8ae51b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:13.413877 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:13.415638 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737a0e06-5e63-4eea-9ce8-c7930a8ae51b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:13.417746 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737a0e06-5e63-4eea-9ce8-c7930a8ae51b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:13.435317 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-737a0e06-5e63-4eea-9ce8-c7930a8ae51b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:13.435317 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:13.435317 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:13.435317 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:13.435317 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:13.435317 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:13.435317 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:13.435317 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:13.435317 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:13.435317 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:13.435317 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:13.435317 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:13.435317 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 23:02:13.435317 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 23:02:13.435317 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 23:02:13.436319 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 23:02:13.436319 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:13.436319 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:13.436319 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:13.436319 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:13.436319 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:13.436319 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:13.436319 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:13.436319 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:13.436319 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:13.436319 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:13.436319 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:13.436319 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:13.436319 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:13.436319 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:13.436319 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:13.436319 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:13.437217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 23:02:13.437217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 23:02:13.437217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 23:02:13.437217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 23:02:13.437217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 23:02:13.437217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 23:02:13.437217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:13.437217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:13.439378 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 364/711] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:02:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:13.445272 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737a0e06-5e63-4eea-9ce8-c7930a8ae51b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:13.507451 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c7c3d-121a-4612-bf0b-5ceb37419b73 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:13.544924 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711f7ed-158b-41ab-b06a-72003007fce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:13.545878 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711f7ed-158b-41ab-b06a-72003007fce2 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:13.546253 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711f7ed-158b-41ab-b06a-72003007fce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:13.546621 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711f7ed-158b-41ab-b06a-72003007fce2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:13.558442 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6711f7ed-158b-41ab-b06a-72003007fce2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:13.560118 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6711f7ed-158b-41ab-b06a-72003007fce2 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:13.566511 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737a0e06-5e63-4eea-9ce8-c7930a8ae51b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:13.579956 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-6711f7ed-158b-41ab-b06a-72003007fce2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:13.579956 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:13.579956 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:13.579956 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:13.579956 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:13.579956 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:13.579956 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:13.579956 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:13.579956 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:13.579956 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:13.579956 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:13.579956 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:13.579956 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:13.579956 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:13.579956 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:13.581107 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:13.581107 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:13.581107 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:13.581107 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:13.581107 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:13.581107 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:13.581107 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:13.581107 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:13.581107 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:13.581107 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:13.581107 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:13.581107 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:13.581107 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:13.581107 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:13.581107 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:13.581107 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:13.581107 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:13.582030 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:13.582030 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:13.582030 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:13.582030 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:13.587465 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 348/712] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:13.588562 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711f7ed-158b-41ab-b06a-72003007fce2 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:13.609476 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bcbba0-87c8-4692-9faf-404d56d969e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:13.610582 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bcbba0-87c8-4692-9faf-404d56d969e1 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:13.611048 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bcbba0-87c8-4692-9faf-404d56d969e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:13.611556 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bcbba0-87c8-4692-9faf-404d56d969e1 None admin] PATH_INFO: `/v3/services/network` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:13.612505 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bcbba0-87c8-4692-9faf-404d56d969e1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:13.614078 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bcbba0-87c8-4692-9faf-404d56d969e1 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:13.628393 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-c3bcbba0-87c8-4692-9faf-404d56d969e1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 23:02:13.628393 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:13.628393 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 23:02:13.628393 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 23:02:13.628393 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 23:02:13.628393 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 23:02:13.628393 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 23:02:13.628393 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:13.628393 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 23:02:13.628393 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:13.628393 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 23:02:13.628393 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:13.628393 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:13.628393 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:13.628393 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:13.628393 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:13.628393 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:13.629551 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:13.629551 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:13.629551 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:13.629551 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:13.629551 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:13.629551 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:13.629551 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 23:02:13.629551 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 23:02:13.629551 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 23:02:13.629551 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 23:02:13.629551 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:13.629551 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:13.629551 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:13.629551 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:13.629551 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:13.629551 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:13.630381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:13.630381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:13.630381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:13.630381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:13.630381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:13.630381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:13.630381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:13.630381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:13.630381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:13.630381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:13.630381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 23:02:13.630381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:13.630381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 23:02:13.630381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:13.633520 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 365/713] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:02:13 2022] GET /identity/v3/services/network => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:13.634620 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bcbba0-87c8-4692-9faf-404d56d969e1 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:13.638004 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:13.660476 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:13.700670 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711f7ed-158b-41ab-b06a-72003007fce2 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:13.738197 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26261e6d-6ea2-4a76-a608-804f61afd329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:13.739063 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26261e6d-6ea2-4a76-a608-804f61afd329 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:13.739336 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26261e6d-6ea2-4a76-a608-804f61afd329 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:13.739713 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26261e6d-6ea2-4a76-a608-804f61afd329 None admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:13.740749 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26261e6d-6ea2-4a76-a608-804f61afd329 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:13.740863 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26261e6d-6ea2-4a76-a608-804f61afd329 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:13.743013 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26261e6d-6ea2-4a76-a608-804f61afd329 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:13.760908 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 349/714] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jul 13 23:02:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:13.760908 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26261e6d-6ea2-4a76-a608-804f61afd329 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:13.764100 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bcbba0-87c8-4692-9faf-404d56d969e1 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:13.811083 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a918148b-8887-4097-859a-197d86ab7e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:13.811796 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a918148b-8887-4097-859a-197d86ab7e47 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:13.812076 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a918148b-8887-4097-859a-197d86ab7e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:13.812347 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a918148b-8887-4097-859a-197d86ab7e47 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:13.813079 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a918148b-8887-4097-859a-197d86ab7e47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:13.813377 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a918148b-8887-4097-859a-197d86ab7e47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:13.814817 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a918148b-8887-4097-859a-197d86ab7e47 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:13.833937 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 366/715] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:13.835277 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a918148b-8887-4097-859a-197d86ab7e47 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:13.894814 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26261e6d-6ea2-4a76-a608-804f61afd329 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:13.922379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:13.931657 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7faea6-c69f-481f-837a-6d6349220947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:13.932325 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7faea6-c69f-481f-837a-6d6349220947 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:13.932601 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7faea6-c69f-481f-837a-6d6349220947 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:13.932875 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7faea6-c69f-481f-837a-6d6349220947 None admin] PATH_INFO: `/v3/services` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:13.933590 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7faea6-c69f-481f-837a-6d6349220947 None admin] RBAC: Adding query filter params (name=network) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:13.933890 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7faea6-c69f-481f-837a-6d6349220947 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:13.935283 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7faea6-c69f-481f-837a-6d6349220947 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:13.940628 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 350/716] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Jul 13 23:02:13 2022] GET /identity/v3/services?name=network => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:13.941655 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7faea6-c69f-481f-837a-6d6349220947 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:13.946003 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:13.992805 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a918148b-8887-4097-859a-197d86ab7e47 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:14.032994 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8eec6c-b21c-49a1-855e-3a16a2793b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:14.034141 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8eec6c-b21c-49a1-855e-3a16a2793b7a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:14.034656 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8eec6c-b21c-49a1-855e-3a16a2793b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:14.035136 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8eec6c-b21c-49a1-855e-3a16a2793b7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:14.036248 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8eec6c-b21c-49a1-855e-3a16a2793b7a None admin] RBAC: Adding query filter params (role.id=8cfd0b6e71e14e0983c3eb4d7478e027, scope.project.id=ab0a5581e4e444c293815e1304e20c05, user.id=4789384d158b4995bd30e1e37e30f5f1) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:14.036874 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8eec6c-b21c-49a1-855e-3a16a2793b7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:14.039126 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8eec6c-b21c-49a1-855e-3a16a2793b7a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:14.051566 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7faea6-c69f-481f-837a-6d6349220947 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:14.052341 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 367/717] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:02:13 2022] GET /identity/v3/role_assignments?user.id=4789384d158b4995bd30e1e37e30f5f1&scope.project.id=ab0a5581e4e444c293815e1304e20c05&role.id=8cfd0b6e71e14e0983c3eb4d7478e027 => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:14.053624 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8eec6c-b21c-49a1-855e-3a16a2793b7a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:14.099571 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd711c-81b3-4991-aa32-9b8ab5c179a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:14.099571 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd711c-81b3-4991-aa32-9b8ab5c179a7 None admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:14.099571 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd711c-81b3-4991-aa32-9b8ab5c179a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:14.099571 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd711c-81b3-4991-aa32-9b8ab5c179a7 None admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:14.100018 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bd711c-81b3-4991-aa32-9b8ab5c179a7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:14.100018 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bd711c-81b3-4991-aa32-9b8ab5c179a7 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:14.127888 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 351/718] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Jul 13 23:02:13 2022] POST /identity/v3/users => generated 254 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:02:14.206068 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d67aa11-1f96-495a-8cbd-7e003c791437 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:14.206390 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d67aa11-1f96-495a-8cbd-7e003c791437 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:14.206620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d67aa11-1f96-495a-8cbd-7e003c791437 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:14.209559 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 352/719] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:02:14.214553 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f442b61-aca7-4309-9337-adddcaeade8f None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:14.215963 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f442b61-aca7-4309-9337-adddcaeade8f None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:14.216256 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f442b61-aca7-4309-9337-adddcaeade8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:14.239490 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-3f442b61-aca7-4309-9337-adddcaeade8f None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:14.241218 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-3f442b61-aca7-4309-9337-adddcaeade8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:14.253596 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8eec6c-b21c-49a1-855e-3a16a2793b7a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:14.307952 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:14.324446 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b89a6-d2c5-4d2b-88cf-e042fc4f68d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:14.325148 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b89a6-d2c5-4d2b-88cf-e042fc4f68d7 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:14.325430 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b89a6-d2c5-4d2b-88cf-e042fc4f68d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:14.325704 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b89a6-d2c5-4d2b-88cf-e042fc4f68d7 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:14.326461 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8b89a6-d2c5-4d2b-88cf-e042fc4f68d7 None admin] RBAC: Adding query filter params (type=network) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:14.326769 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8b89a6-d2c5-4d2b-88cf-e042fc4f68d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:14.333060 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8b89a6-d2c5-4d2b-88cf-e042fc4f68d7 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:14.340578 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:14.344272 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 368/720] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Jul 13 23:02:14 2022] GET /identity/v3/services?type=network => generated 124 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:14.386782 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 353/721] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 23:02:14.427321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b89a6-d2c5-4d2b-88cf-e042fc4f68d7 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:14.601868 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b89a6-d2c5-4d2b-88cf-e042fc4f68d7 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:14.648838 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09057dd-c48c-49c2-9cf1-fc08d08ee6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:14.650006 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09057dd-c48c-49c2-9cf1-fc08d08ee6a7 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:14.650520 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09057dd-c48c-49c2-9cf1-fc08d08ee6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:14.651000 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09057dd-c48c-49c2-9cf1-fc08d08ee6a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:14.667890 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09057dd-c48c-49c2-9cf1-fc08d08ee6a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:02:14.676324 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-b09057dd-c48c-49c2-9cf1-fc08d08ee6a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:02:14.676324 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:14.676324 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:14.676324 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:14.676324 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:14.676324 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:14.676324 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:14.676324 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:14.676324 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:14.676324 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:14.676324 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:14.676324 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:14.676324 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:02:14.676324 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:02:14.676324 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:02:14.676324 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:02:14.677321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:02:14.677321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:02:14.677321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:02:14.677321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:02:14.677321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:14.677321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:14.677321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:14.677321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:14.677321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:14.677321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:14.677321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:14.677321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:14.677321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:14.677321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:14.677321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:14.677321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:14.678190 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:14.678190 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:14.678190 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:14.678190 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:14.678190 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:02:14.678190 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:02:14.678190 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:02:14.678190 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:02:14.678190 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:02:14.678190 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:02:14.678190 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:02:14.678190 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:14.682485 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 369/722] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jul 13 23:02:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:14.690122 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f442b61-aca7-4309-9337-adddcaeade8f None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:14.836126 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f442b61-aca7-4309-9337-adddcaeade8f None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:14.888455 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a760de0-3b90-4eb8-8c48-2ea250e09e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:14.889155 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a760de0-3b90-4eb8-8c48-2ea250e09e00 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:14.889440 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a760de0-3b90-4eb8-8c48-2ea250e09e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:14.889716 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a760de0-3b90-4eb8-8c48-2ea250e09e00 None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:14.890489 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a760de0-3b90-4eb8-8c48-2ea250e09e00 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:14.890802 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a760de0-3b90-4eb8-8c48-2ea250e09e00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:14.892247 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a760de0-3b90-4eb8-8c48-2ea250e09e00 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:14.907497 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09057dd-c48c-49c2-9cf1-fc08d08ee6a7 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:14.908021 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 354/723] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:02:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:15.028096 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:15.068699 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09057dd-c48c-49c2-9cf1-fc08d08ee6a7 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:15.106211 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5eae8-b3e0-4140-969a-e7bb4d082ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:15.107026 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5eae8-b3e0-4140-969a-e7bb4d082ab2 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:15.107309 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5eae8-b3e0-4140-969a-e7bb4d082ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:15.107610 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5eae8-b3e0-4140-969a-e7bb4d082ab2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:15.125586 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d5eae8-b3e0-4140-969a-e7bb4d082ab2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:15.128189 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d5eae8-b3e0-4140-969a-e7bb4d082ab2 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:15.140600 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e48cb-a2b7-46bd-8fa5-c2dfbadfce16 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:15.140900 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e48cb-a2b7-46bd-8fa5-c2dfbadfce16 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:15.141084 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e48cb-a2b7-46bd-8fa5-c2dfbadfce16 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:15.142755 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 355/724] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:15.148357 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a79a49-312c-417b-ac37-b603b48276c0 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:15.148647 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a79a49-312c-417b-ac37-b603b48276c0 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:15.148930 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a79a49-312c-417b-ac37-b603b48276c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:15.152728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-58d5eae8-b3e0-4140-969a-e7bb4d082ab2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:15.152728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:15.152728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:15.152728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:15.152728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:15.152728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:15.152728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:15.152728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:15.152728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:15.152728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:15.152728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:15.152728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:15.152728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:15.152728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:15.152728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:15.153594 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:15.153594 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:15.153594 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:15.153594 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:15.153594 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:15.153594 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:15.153594 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:15.153594 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:15.153594 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:15.153594 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:15.153594 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:15.153594 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:15.153594 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:15.153594 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:15.153594 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:15.153594 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:15.153594 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:15.154561 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:15.154561 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:15.154561 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:15.154561 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:15.155243 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 370/725] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:15.167161 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5eae8-b3e0-4140-969a-e7bb4d082ab2 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:15.177615 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-04a79a49-312c-417b-ac37-b603b48276c0 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:15.179403 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-04a79a49-312c-417b-ac37-b603b48276c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:15.212285 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:15.268176 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 356/726] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 23:02:15.294997 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a79a49-312c-417b-ac37-b603b48276c0 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:15.382584 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5eae8-b3e0-4140-969a-e7bb4d082ab2 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:15.387902 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:15.460763 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08098839-5c18-431e-85ba-a77b2bc53a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:15.461355 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08098839-5c18-431e-85ba-a77b2bc53a50 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:15.461636 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08098839-5c18-431e-85ba-a77b2bc53a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:15.461909 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08098839-5c18-431e-85ba-a77b2bc53a50 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:15.462648 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08098839-5c18-431e-85ba-a77b2bc53a50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:15.462954 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08098839-5c18-431e-85ba-a77b2bc53a50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:15.464383 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08098839-5c18-431e-85ba-a77b2bc53a50 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:15.488656 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a79a49-312c-417b-ac37-b603b48276c0 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:15.495243 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 371/727] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:15.496328 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08098839-5c18-431e-85ba-a77b2bc53a50 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:15.562856 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706258e5-f5b1-4e6f-9194-a0c342142657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:15.563553 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706258e5-f5b1-4e6f-9194-a0c342142657 None admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:15.563842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706258e5-f5b1-4e6f-9194-a0c342142657 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:15.564115 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706258e5-f5b1-4e6f-9194-a0c342142657 None admin] PATH_INFO: `/v3/services` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:15.564865 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706258e5-f5b1-4e6f-9194-a0c342142657 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:15.566302 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706258e5-f5b1-4e6f-9194-a0c342142657 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:15.600099 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 357/728] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Jul 13 23:02:15 2022] POST /identity/v3/services => generated 251 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:02:15.683741 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08098839-5c18-431e-85ba-a77b2bc53a50 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:15.748959 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b59fcd-6a71-4efe-bd43-aaae5a0618c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:15.750134 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b59fcd-6a71-4efe-bd43-aaae5a0618c2 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:15.750634 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b59fcd-6a71-4efe-bd43-aaae5a0618c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:15.751113 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b59fcd-6a71-4efe-bd43-aaae5a0618c2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:15.773545 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b59fcd-6a71-4efe-bd43-aaae5a0618c2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:15.775426 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b59fcd-6a71-4efe-bd43-aaae5a0618c2 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:15.796327 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-21b59fcd-6a71-4efe-bd43-aaae5a0618c2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 13 23:02:15.796327 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:15.796327 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:15.796327 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:15.796327 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:15.796327 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:15.796327 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:15.796327 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:15.796327 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:15.796327 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:15.796327 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:15.796327 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:15.796327 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 23:02:15.796327 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 23:02:15.796327 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 23:02:15.796327 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 23:02:15.797342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:15.797342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:15.797342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 23:02:15.797342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:15.797342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 23:02:15.797342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:15.797342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:15.797342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:15.797342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:15.797342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:15.797342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:15.797342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:15.797342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:15.797342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:15.797342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:15.797342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:15.797342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:15.797342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:15.798267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:15.798267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:15.798267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 23:02:15.798267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 23:02:15.798267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 23:02:15.798267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 23:02:15.798267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 23:02:15.798267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 23:02:15.798267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 13 23:02:15.798267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:15.801072 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 372/729] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jul 13 23:02:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:15.807131 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706258e5-f5b1-4e6f-9194-a0c342142657 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:15.862066 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:15.943563 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706258e5-f5b1-4e6f-9194-a0c342142657 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:15.981761 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8dc679-cc57-41bc-9a5d-0daf317413bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:15.982439 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8dc679-cc57-41bc-9a5d-0daf317413bf None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:15.982719 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8dc679-cc57-41bc-9a5d-0daf317413bf None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:15.982989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8dc679-cc57-41bc-9a5d-0daf317413bf None admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:15.983870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8dc679-cc57-41bc-9a5d-0daf317413bf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:15.984180 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8dc679-cc57-41bc-9a5d-0daf317413bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:15.985665 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8dc679-cc57-41bc-9a5d-0daf317413bf None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:16.005276 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 358/730] 10.222.0.60 () {56 vars in 1135 bytes} [Wed Jul 13 23:02:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:16.012114 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b59fcd-6a71-4efe-bd43-aaae5a0618c2 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:16.041033 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:16.176106 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b59fcd-6a71-4efe-bd43-aaae5a0618c2 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:16.225962 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3ad74-9e84-42b1-8b55-2c5aa27a9d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:16.226624 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3ad74-9e84-42b1-8b55-2c5aa27a9d2c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:16.226907 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3ad74-9e84-42b1-8b55-2c5aa27a9d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:16.227183 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3ad74-9e84-42b1-8b55-2c5aa27a9d2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:16.335496 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3ad74-9e84-42b1-8b55-2c5aa27a9d2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:16.337934 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a3ad74-9e84-42b1-8b55-2c5aa27a9d2c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:16.380173 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:16.546281 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 373/731] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:02:16 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 13 23:02:16.552621 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8dc679-cc57-41bc-9a5d-0daf317413bf None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:16.787502 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:16.823262 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8dc679-cc57-41bc-9a5d-0daf317413bf None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:16.904688 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d37de34-2e28-4af2-9606-543ed5737f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:16.906044 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37de34-2e28-4af2-9606-543ed5737f9c None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:16.906516 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37de34-2e28-4af2-9606-543ed5737f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:16.907014 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37de34-2e28-4af2-9606-543ed5737f9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:16.930624 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d37de34-2e28-4af2-9606-543ed5737f9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:16.933268 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d37de34-2e28-4af2-9606-543ed5737f9c None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:16.979954 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-9d37de34-2e28-4af2-9606-543ed5737f9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:16.979954 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:16.979954 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:16.979954 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:16.979954 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:16.979954 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:16.979954 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:16.979954 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:16.979954 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:16.979954 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:16.979954 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:16.979954 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:16.979954 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:16.979954 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:16.979954 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:16.980915 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:16.980915 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:16.980915 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:16.980915 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:16.980915 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:16.980915 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:16.980915 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:16.980915 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:16.980915 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:16.980915 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:16.980915 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:16.980915 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:16.980915 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:16.980915 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:16.980915 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:16.980915 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:16.980915 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:16.981820 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:16.981820 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:16.981820 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:16.981820 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:16.982674 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 359/732] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 431 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:02:16.988079 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3ad74-9e84-42b1-8b55-2c5aa27a9d2c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:17.040092 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855882d-af94-4e04-bd3d-02394c9455f4 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:17.040855 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855882d-af94-4e04-bd3d-02394c9455f4 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:17.041542 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855882d-af94-4e04-bd3d-02394c9455f4 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:17.048322 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 360/733] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:17.050965 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aac0f2-efa7-44ba-88a5-fa0a0a9d32d3 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:17.051740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aac0f2-efa7-44ba-88a5-fa0a0a9d32d3 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:17.052423 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aac0f2-efa7-44ba-88a5-fa0a0a9d32d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:17.085888 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-95aac0f2-efa7-44ba-88a5-fa0a0a9d32d3 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:17.087180 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-95aac0f2-efa7-44ba-88a5-fa0a0a9d32d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:17.201184 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a3ad74-9e84-42b1-8b55-2c5aa27a9d2c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:17.213182 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 361/734] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:17 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:02:17.235899 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aac0f2-efa7-44ba-88a5-fa0a0a9d32d3 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:17.240138 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:17.275199 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9c9e84-632e-4bf5-9803-55b6c2ad3d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:17.276017 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c9e84-632e-4bf5-9803-55b6c2ad3d52 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:17.276295 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c9e84-632e-4bf5-9803-55b6c2ad3d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:17.276570 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c9e84-632e-4bf5-9803-55b6c2ad3d52 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:17.277314 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9c9e84-632e-4bf5-9803-55b6c2ad3d52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:17.277621 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9c9e84-632e-4bf5-9803-55b6c2ad3d52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:17.279114 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9c9e84-632e-4bf5-9803-55b6c2ad3d52 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:17.306551 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 374/735] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:02:17.311061 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:17.315778 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9c9e84-632e-4bf5-9803-55b6c2ad3d52 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:17.348038 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aac0f2-efa7-44ba-88a5-fa0a0a9d32d3 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:17.389660 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1a86d0-e188-4204-b108-e1dab27a7692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:17.390520 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a86d0-e188-4204-b108-e1dab27a7692 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:17.390804 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a86d0-e188-4204-b108-e1dab27a7692 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:17.391078 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a86d0-e188-4204-b108-e1dab27a7692 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:17.402727 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1a86d0-e188-4204-b108-e1dab27a7692 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:02:17.410684 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-1d1a86d0-e188-4204-b108-e1dab27a7692 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:02:17.410684 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:17.410684 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:17.410684 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:17.410684 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:17.410684 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:17.410684 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:17.410684 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:17.410684 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:17.410684 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:17.410684 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:17.410684 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:17.410684 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:02:17.410684 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:02:17.410684 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:02:17.410684 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:02:17.413441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:02:17.413441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:02:17.413441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:02:17.413441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:02:17.413441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:17.413441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:17.413441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:17.413441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:17.413441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:17.413441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:17.413441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:17.413441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:17.413441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:17.413441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:17.413441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:17.413441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:17.415188 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:17.415188 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:17.415188 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:17.415188 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:17.415188 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:02:17.415188 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:02:17.415188 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:02:17.415188 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:02:17.415188 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:02:17.415188 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:02:17.415188 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:02:17.415188 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:17.415188 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 362/736] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jul 13 23:02:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:02:17.415188 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863dd13-dce6-4782-a5bc-740458f21383 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:17.417339 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863dd13-dce6-4782-a5bc-740458f21383 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:17.417339 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863dd13-dce6-4782-a5bc-740458f21383 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:17.421123 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 363/737] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:17.422125 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6863dd13-dce6-4782-a5bc-740458f21383 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:17.517610 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9c9e84-632e-4bf5-9803-55b6c2ad3d52 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:17.536951 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6863dd13-dce6-4782-a5bc-740458f21383 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:17.567681 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:17.577901 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9861aa4-09eb-4f3c-91b5-9179191cfc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:17.578480 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9861aa4-09eb-4f3c-91b5-9179191cfc26 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:17.578768 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9861aa4-09eb-4f3c-91b5-9179191cfc26 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:17.579047 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9861aa4-09eb-4f3c-91b5-9179191cfc26 None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:17.579908 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9861aa4-09eb-4f3c-91b5-9179191cfc26 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:17.580295 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9861aa4-09eb-4f3c-91b5-9179191cfc26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:17.581694 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9861aa4-09eb-4f3c-91b5-9179191cfc26 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:17.587916 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5786d2-5940-4340-bf43-f59de44bf05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:17.589544 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5786d2-5940-4340-bf43-f59de44bf05c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:17.590223 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5786d2-5940-4340-bf43-f59de44bf05c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:17.590862 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5786d2-5940-4340-bf43-f59de44bf05c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:17.597889 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 364/738] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:02:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:17.599412 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d6359-0fb4-40d7-b54d-96a18d6aaa44 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:17.599733 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d6359-0fb4-40d7-b54d-96a18d6aaa44 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:17.600073 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d6359-0fb4-40d7-b54d-96a18d6aaa44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:17.605740 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5786d2-5940-4340-bf43-f59de44bf05c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:17.608834 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5786d2-5940-4340-bf43-f59de44bf05c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:17.618898 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-667d6359-0fb4-40d7-b54d-96a18d6aaa44 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:17.620156 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-667d6359-0fb4-40d7-b54d-96a18d6aaa44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:17.638469 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-4f5786d2-5940-4340-bf43-f59de44bf05c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:17.638469 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:17.638469 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:17.638469 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:17.638469 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:17.638469 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:17.638469 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:17.638469 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:17.638469 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:17.638469 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:17.638469 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:17.638469 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:17.638469 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 23:02:17.638469 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 23:02:17.638469 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 23:02:17.640426 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 23:02:17.640426 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:17.640426 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:17.640426 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:17.640426 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:17.640426 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:17.640426 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:17.640426 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:17.640426 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:17.640426 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:17.640426 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:17.640426 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:17.640426 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:17.640426 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:17.640426 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:17.640426 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:17.640426 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:17.642117 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 23:02:17.642117 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 23:02:17.642117 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 23:02:17.642117 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 23:02:17.642117 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 23:02:17.642117 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 23:02:17.642117 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:17.642117 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:17.646217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 375/739] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:02:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:02:17.648046 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5786d2-5940-4340-bf43-f59de44bf05c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:17.703456 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 365/740] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:17 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:02:17.704423 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d6359-0fb4-40d7-b54d-96a18d6aaa44 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:17.730999 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:17.821742 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d6359-0fb4-40d7-b54d-96a18d6aaa44 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:17.842957 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5786d2-5940-4340-bf43-f59de44bf05c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:17.845678 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:17.863494 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d01c0-8e26-46eb-a181-6faafbef25fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:17.864571 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d01c0-8e26-46eb-a181-6faafbef25fb None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:17.865012 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d01c0-8e26-46eb-a181-6faafbef25fb None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:17.865436 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d01c0-8e26-46eb-a181-6faafbef25fb None admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:17.866931 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d01c0-8e26-46eb-a181-6faafbef25fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:17.867410 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d01c0-8e26-46eb-a181-6faafbef25fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:17.869225 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d01c0-8e26-46eb-a181-6faafbef25fb None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:17.883831 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 366/741] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jul 13 23:02:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:17.884933 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d01c0-8e26-46eb-a181-6faafbef25fb None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:17.912862 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50185bce-19cb-476b-a0cb-2f77646876d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:17.914592 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50185bce-19cb-476b-a0cb-2f77646876d4 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:17.915190 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50185bce-19cb-476b-a0cb-2f77646876d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:17.915813 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50185bce-19cb-476b-a0cb-2f77646876d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:17.936126 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50185bce-19cb-476b-a0cb-2f77646876d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:17.939195 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50185bce-19cb-476b-a0cb-2f77646876d4 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:17.976277 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-50185bce-19cb-476b-a0cb-2f77646876d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:17.976277 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:17.976277 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:17.976277 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:17.976277 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:17.976277 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:17.976277 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:17.976277 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:17.976277 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:17.976277 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:17.976277 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:17.976277 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:17.976277 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:17.976277 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:17.976277 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:17.977265 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:17.977265 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:17.977265 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:17.977265 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:17.977265 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:17.977265 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:17.977265 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:17.977265 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:17.977265 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:17.977265 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:17.977265 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:17.977265 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:17.977265 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:17.977265 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:17.977265 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:17.977265 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:17.977265 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:17.978176 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:17.978176 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:17.978176 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:17.978176 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:17.983935 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 376/742] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:17.983935 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50185bce-19cb-476b-a0cb-2f77646876d4 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:18.007687 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d01c0-8e26-46eb-a181-6faafbef25fb None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:18.046476 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6628fa5-7fb4-4787-857b-8925987ebec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:18.047787 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6628fa5-7fb4-4787-857b-8925987ebec4 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:18.048359 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6628fa5-7fb4-4787-857b-8925987ebec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:18.048851 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6628fa5-7fb4-4787-857b-8925987ebec4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:18.061824 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6628fa5-7fb4-4787-857b-8925987ebec4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:02:18.070835 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-c6628fa5-7fb4-4787-857b-8925987ebec4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:02:18.070835 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:18.070835 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:18.070835 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:18.070835 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:18.070835 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:18.070835 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:18.070835 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:18.070835 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:18.070835 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:18.070835 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:18.070835 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:18.070835 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:02:18.070835 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:02:18.070835 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:02:18.070835 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:02:18.074020 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:02:18.074020 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:02:18.074020 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:02:18.074020 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:02:18.074020 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:18.074020 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:18.074020 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:18.074020 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:18.074020 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:18.074020 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:18.074020 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:18.074020 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:18.074020 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:18.074020 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:18.074020 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:18.074020 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:18.074927 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:18.074927 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:18.074927 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:18.074927 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:18.074927 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:02:18.074927 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:02:18.074927 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:02:18.074927 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:02:18.074927 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:02:18.074927 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:02:18.074927 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:02:18.074927 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:18.076235 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 367/743] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jul 13 23:02:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:18.077351 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6628fa5-7fb4-4787-857b-8925987ebec4 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:18.137530 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:18.167338 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:18.187349 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50185bce-19cb-476b-a0cb-2f77646876d4 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:18.253431 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f260d73f-c8de-4aea-a1ea-74289dcab2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:18.254570 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f260d73f-c8de-4aea-a1ea-74289dcab2e8 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:18.255055 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f260d73f-c8de-4aea-a1ea-74289dcab2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:18.255569 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f260d73f-c8de-4aea-a1ea-74289dcab2e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:18.257445 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f260d73f-c8de-4aea-a1ea-74289dcab2e8 None admin] RBAC: Adding query filter params (role.id=8cfd0b6e71e14e0983c3eb4d7478e027, scope.project.id=ab0a5581e4e444c293815e1304e20c05, user.id=786fc44611804d3ca6280ffcab415569) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:18.257999 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f260d73f-c8de-4aea-a1ea-74289dcab2e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:18.260816 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6628fa5-7fb4-4787-857b-8925987ebec4 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:18.261051 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f260d73f-c8de-4aea-a1ea-74289dcab2e8 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:18.274018 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 377/744] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:02:17 2022] GET /identity/v3/role_assignments?user.id=786fc44611804d3ca6280ffcab415569&scope.project.id=ab0a5581e4e444c293815e1304e20c05&role.id=8cfd0b6e71e14e0983c3eb4d7478e027 => generated 259 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:18.275563 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f260d73f-c8de-4aea-a1ea-74289dcab2e8 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:18.331697 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d21eac-1bfe-42c6-aa7f-edc9bc8ab133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:18.332778 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d21eac-1bfe-42c6-aa7f-edc9bc8ab133 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:18.333287 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d21eac-1bfe-42c6-aa7f-edc9bc8ab133 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:18.333780 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d21eac-1bfe-42c6-aa7f-edc9bc8ab133 None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:18.335398 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d21eac-1bfe-42c6-aa7f-edc9bc8ab133 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:18.335902 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d21eac-1bfe-42c6-aa7f-edc9bc8ab133 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:18.338624 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d21eac-1bfe-42c6-aa7f-edc9bc8ab133 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:18.363644 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 368/745] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:18.365763 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17669cc-5c95-4834-8fc1-ba9c67212be1 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:18.366850 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17669cc-5c95-4834-8fc1-ba9c67212be1 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:18.367341 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17669cc-5c95-4834-8fc1-ba9c67212be1 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:18.370659 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 369/746] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:18.379780 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438581e8-1b1a-4f4a-a2ce-e2f17dc7a326 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:18.380240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438581e8-1b1a-4f4a-a2ce-e2f17dc7a326 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:18.380715 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438581e8-1b1a-4f4a-a2ce-e2f17dc7a326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:18.418260 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-438581e8-1b1a-4f4a-a2ce-e2f17dc7a326 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:18.421220 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-438581e8-1b1a-4f4a-a2ce-e2f17dc7a326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:18.510617 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f260d73f-c8de-4aea-a1ea-74289dcab2e8 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:18.566154 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 370/747] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:18 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 23:02:18.567904 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438581e8-1b1a-4f4a-a2ce-e2f17dc7a326 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:18.594484 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae63b9-a63e-4183-94de-ccbbedf62390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:18.596270 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae63b9-a63e-4183-94de-ccbbedf62390 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:18.597273 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae63b9-a63e-4183-94de-ccbbedf62390 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:18.598076 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae63b9-a63e-4183-94de-ccbbedf62390 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:18.599835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae63b9-a63e-4183-94de-ccbbedf62390 None admin] RBAC: Adding query filter params (name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:18.601679 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae63b9-a63e-4183-94de-ccbbedf62390 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:18.604636 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae63b9-a63e-4183-94de-ccbbedf62390 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:18.621638 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 378/748] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jul 13 23:02:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:18.623421 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:18.624059 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae63b9-a63e-4183-94de-ccbbedf62390 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:18.666280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:18.720117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438581e8-1b1a-4f4a-a2ce-e2f17dc7a326 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:18.759430 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db025b1-9315-4c4a-a75d-750be87351ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:18.760274 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db025b1-9315-4c4a-a75d-750be87351ba None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:18.760551 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db025b1-9315-4c4a-a75d-750be87351ba None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:18.760821 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db025b1-9315-4c4a-a75d-750be87351ba None admin] PATH_INFO: `/v3/users/nova` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:18.768753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db025b1-9315-4c4a-a75d-750be87351ba None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:18.770241 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db025b1-9315-4c4a-a75d-750be87351ba None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:18.785919 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-6db025b1-9315-4c4a-a75d-750be87351ba None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 23:02:18.785919 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:18.785919 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:18.785919 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:18.785919 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:18.785919 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:18.785919 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:18.785919 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:18.785919 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:18.785919 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:18.785919 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:18.785919 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:18.785919 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 23:02:18.785919 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 23:02:18.785919 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 23:02:18.785919 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 23:02:18.786924 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:18.786924 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:18.786924 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 23:02:18.786924 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:18.786924 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 23:02:18.786924 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:18.786924 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:18.786924 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:18.786924 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:18.786924 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:18.786924 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:18.786924 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:18.786924 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:18.786924 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:18.786924 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:18.786924 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:18.786924 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:18.786924 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:18.788533 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:18.788533 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:18.788533 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 23:02:18.788533 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 23:02:18.788533 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 23:02:18.788533 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 23:02:18.788533 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 23:02:18.788533 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 23:02:18.788533 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 23:02:18.788533 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:18.788533 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 371/749] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Jul 13 23:02:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:18.788533 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db025b1-9315-4c4a-a75d-750be87351ba None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:18.802593 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae63b9-a63e-4183-94de-ccbbedf62390 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:18.843064 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a14f14-38dd-4265-99ff-e8ea73eefe81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:18.843806 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a14f14-38dd-4265-99ff-e8ea73eefe81 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:18.844090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a14f14-38dd-4265-99ff-e8ea73eefe81 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:18.844370 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a14f14-38dd-4265-99ff-e8ea73eefe81 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:18.845163 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a14f14-38dd-4265-99ff-e8ea73eefe81 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:18.846509 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a14f14-38dd-4265-99ff-e8ea73eefe81 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:18.863155 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 379/750] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Jul 13 23:02:18 2022] POST /identity/v3/services => generated 240 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:02:18.877128 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65274e8a-6cb0-454b-a11f-95b8aaa20f51 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:18.877457 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65274e8a-6cb0-454b-a11f-95b8aaa20f51 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:18.877689 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65274e8a-6cb0-454b-a11f-95b8aaa20f51 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:18.879287 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 380/751] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:18 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:18.879873 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65274e8a-6cb0-454b-a11f-95b8aaa20f51 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:18.901154 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:18.920537 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db025b1-9315-4c4a-a75d-750be87351ba None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:18.991653 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb8c3b2-1c7e-4bcf-9a91-5c19fa4ce3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:18.992893 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb8c3b2-1c7e-4bcf-9a91-5c19fa4ce3c9 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:18.993441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb8c3b2-1c7e-4bcf-9a91-5c19fa4ce3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:18.993936 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb8c3b2-1c7e-4bcf-9a91-5c19fa4ce3c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:18.999403 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:19.013680 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb8c3b2-1c7e-4bcf-9a91-5c19fa4ce3c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:19.016038 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb8c3b2-1c7e-4bcf-9a91-5c19fa4ce3c9 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:19.040492 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65274e8a-6cb0-454b-a11f-95b8aaa20f51 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:19.040837 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-2eb8c3b2-1c7e-4bcf-9a91-5c19fa4ce3c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:19.040837 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:19.040837 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:19.040837 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:19.040837 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:19.040837 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:19.040837 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:19.040837 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:19.040837 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:19.040837 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:19.040837 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:19.040837 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:19.040837 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:19.040837 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:19.040837 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:19.041668 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:19.041668 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:19.041668 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:19.041668 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:19.041668 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:19.041668 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:19.041668 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:19.041668 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:19.041668 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:19.041668 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:19.041668 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:19.041668 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:19.041668 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:19.041668 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:19.041668 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:19.041668 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:19.041668 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:19.042537 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:19.042537 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:19.042537 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:19.042537 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:19.047022 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 372/752] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:19.048575 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf511192-55bf-4952-9b4d-61ff8118cc7e None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:19.048860 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf511192-55bf-4952-9b4d-61ff8118cc7e None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:19.049140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf511192-55bf-4952-9b4d-61ff8118cc7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:19.074844 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-cf511192-55bf-4952-9b4d-61ff8118cc7e None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:19.076156 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-cf511192-55bf-4952-9b4d-61ff8118cc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:19.094563 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd06869-7980-4627-8a39-38d96ddc97c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:19.095616 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd06869-7980-4627-8a39-38d96ddc97c7 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:19.095872 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd06869-7980-4627-8a39-38d96ddc97c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:19.096152 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd06869-7980-4627-8a39-38d96ddc97c7 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:19.097195 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd06869-7980-4627-8a39-38d96ddc97c7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:19.097524 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd06869-7980-4627-8a39-38d96ddc97c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:19.099398 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd06869-7980-4627-8a39-38d96ddc97c7 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:19.123071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 381/753] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Jul 13 23:02:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:19.124036 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd06869-7980-4627-8a39-38d96ddc97c7 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:19.182460 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 373/754] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:02:19.183409 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf511192-55bf-4952-9b4d-61ff8118cc7e None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:19.256173 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:19.279302 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd06869-7980-4627-8a39-38d96ddc97c7 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:19.296353 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf511192-55bf-4952-9b4d-61ff8118cc7e None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:19.330942 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8657dcb1-e091-4aa9-85a3-50009caa748c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:19.331955 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8657dcb1-e091-4aa9-85a3-50009caa748c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:19.332388 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8657dcb1-e091-4aa9-85a3-50009caa748c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:19.332811 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8657dcb1-e091-4aa9-85a3-50009caa748c None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:19.333804 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8657dcb1-e091-4aa9-85a3-50009caa748c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:19.334291 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8657dcb1-e091-4aa9-85a3-50009caa748c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:19.336042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8657dcb1-e091-4aa9-85a3-50009caa748c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:19.338162 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:19.344205 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1651193e-611d-42d2-a79e-0a4c980b7580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:19.345125 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651193e-611d-42d2-a79e-0a4c980b7580 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:19.345482 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651193e-611d-42d2-a79e-0a4c980b7580 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:19.345885 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651193e-611d-42d2-a79e-0a4c980b7580 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:19.358259 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 382/755] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:19.359622 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8657dcb1-e091-4aa9-85a3-50009caa748c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:19.372816 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1651193e-611d-42d2-a79e-0a4c980b7580 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:19.374847 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1651193e-611d-42d2-a79e-0a4c980b7580 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:19.398413 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-1651193e-611d-42d2-a79e-0a4c980b7580 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:19.398413 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:19.398413 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:19.398413 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:19.398413 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:19.398413 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:19.398413 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:19.398413 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:19.398413 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:19.398413 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:19.398413 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:19.398413 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:19.398413 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:19.398413 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:19.398413 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:19.399550 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:19.399550 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:19.399550 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:19.399550 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:19.399550 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:19.399550 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:19.399550 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:19.399550 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:19.399550 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:19.399550 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:19.399550 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:19.399550 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:19.399550 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:19.399550 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:19.399550 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:19.399550 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:19.399550 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:19.400687 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:19.400687 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:19.400687 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:19.400687 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:19.407156 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 374/756] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:19.408438 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1651193e-611d-42d2-a79e-0a4c980b7580 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:19.480610 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8657dcb1-e091-4aa9-85a3-50009caa748c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:19.521240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfd5590-f79b-4266-843b-6a3b20dcf106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:19.522274 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd5590-f79b-4266-843b-6a3b20dcf106 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:19.522631 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd5590-f79b-4266-843b-6a3b20dcf106 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:19.523169 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd5590-f79b-4266-843b-6a3b20dcf106 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:19.524425 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfd5590-f79b-4266-843b-6a3b20dcf106 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:19.526125 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfd5590-f79b-4266-843b-6a3b20dcf106 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:19.543437 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1651193e-611d-42d2-a79e-0a4c980b7580 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:19.544081 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-cdfd5590-f79b-4266-843b-6a3b20dcf106 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 23:02:19.544081 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:19.544081 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 23:02:19.544081 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 23:02:19.544081 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 23:02:19.544081 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 23:02:19.544081 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 23:02:19.544081 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:19.544081 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 23:02:19.544081 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:19.544081 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 23:02:19.544081 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:19.544081 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:19.544081 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:19.544081 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:19.544081 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:19.545234 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:19.545234 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:19.545234 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:19.545234 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:19.545234 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:19.545234 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:19.545234 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:19.545234 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 23:02:19.545234 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 23:02:19.545234 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 23:02:19.545234 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 23:02:19.545234 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:19.545234 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:19.545234 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:19.545234 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:19.545234 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:19.546267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:19.546267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:19.546267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:19.546267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:19.546267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:19.546267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:19.546267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:19.546267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:19.546267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:19.546267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:19.546267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:19.546267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 23:02:19.546267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:19.546267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 23:02:19.546267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:19.551564 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 383/757] 10.222.0.60 () {56 vars in 1129 bytes} [Wed Jul 13 23:02:19 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:19.551564 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfd5590-f79b-4266-843b-6a3b20dcf106 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:19.590280 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:19.609317 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57137ff0-bb29-45c1-9386-f7973e18f972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:19.610385 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57137ff0-bb29-45c1-9386-f7973e18f972 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:19.610822 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57137ff0-bb29-45c1-9386-f7973e18f972 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:19.611247 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57137ff0-bb29-45c1-9386-f7973e18f972 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:19.620720 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57137ff0-bb29-45c1-9386-f7973e18f972 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:19.622515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57137ff0-bb29-45c1-9386-f7973e18f972 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:19.640252 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-57137ff0-bb29-45c1-9386-f7973e18f972 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 23:02:19.640252 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:19.640252 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:19.640252 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:19.640252 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:19.640252 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:19.640252 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:19.640252 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:19.640252 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:19.640252 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:19.640252 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:19.640252 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:19.640252 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 23:02:19.640252 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 23:02:19.640252 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 23:02:19.640252 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 23:02:19.641440 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:19.641440 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:19.641440 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 23:02:19.641440 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:19.641440 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 23:02:19.641440 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:19.641440 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:19.641440 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:19.641440 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:19.641440 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:19.641440 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:19.641440 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:19.641440 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:19.641440 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:19.641440 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:19.641440 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:19.641440 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:19.641440 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:19.642628 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:19.642628 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:19.642628 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 23:02:19.642628 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 23:02:19.642628 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 23:02:19.642628 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 23:02:19.642628 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 23:02:19.642628 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 23:02:19.642628 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 23:02:19.642628 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:19.649773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 375/758] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Jul 13 23:02:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:19.650560 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:19.651674 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57137ff0-bb29-45c1-9386-f7973e18f972 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:19.706134 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfd5590-f79b-4266-843b-6a3b20dcf106 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:19.746114 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e00bde-ebcb-4ec3-a6d4-18510debb47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:19.747053 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e00bde-ebcb-4ec3-a6d4-18510debb47c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:19.747489 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e00bde-ebcb-4ec3-a6d4-18510debb47c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:19.747943 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e00bde-ebcb-4ec3-a6d4-18510debb47c None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:19.748866 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e00bde-ebcb-4ec3-a6d4-18510debb47c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:19.749328 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e00bde-ebcb-4ec3-a6d4-18510debb47c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:19.750919 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e00bde-ebcb-4ec3-a6d4-18510debb47c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:19.771599 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 384/759] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:19.771599 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e00bde-ebcb-4ec3-a6d4-18510debb47c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:19.799428 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57137ff0-bb29-45c1-9386-f7973e18f972 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:19.840240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a66068-d3fd-4471-abc8-4cb7de5d957b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:19.840969 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a66068-d3fd-4471-abc8-4cb7de5d957b None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:19.841250 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a66068-d3fd-4471-abc8-4cb7de5d957b None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:19.841525 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a66068-d3fd-4471-abc8-4cb7de5d957b None admin] PATH_INFO: `/v3/services` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:19.842255 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a66068-d3fd-4471-abc8-4cb7de5d957b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:19.842564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a66068-d3fd-4471-abc8-4cb7de5d957b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:19.844379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a66068-d3fd-4471-abc8-4cb7de5d957b None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:19.854702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 376/760] 10.222.0.60 () {56 vars in 1124 bytes} [Wed Jul 13 23:02:19 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:19.854702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a66068-d3fd-4471-abc8-4cb7de5d957b None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:19.917090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e00bde-ebcb-4ec3-a6d4-18510debb47c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:19.927687 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:19.939091 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:19.961626 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26346739-c65e-4a49-be93-706d8739f393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:19.962755 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26346739-c65e-4a49-be93-706d8739f393 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:19.963272 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26346739-c65e-4a49-be93-706d8739f393 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:19.963833 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26346739-c65e-4a49-be93-706d8739f393 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:19.965016 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26346739-c65e-4a49-be93-706d8739f393 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:19.965566 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26346739-c65e-4a49-be93-706d8739f393 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:19.967262 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26346739-c65e-4a49-be93-706d8739f393 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:19.995307 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 385/761] 10.222.0.60 () {56 vars in 1149 bytes} [Wed Jul 13 23:02:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:19.996705 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26346739-c65e-4a49-be93-706d8739f393 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:20.013232 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a66068-d3fd-4471-abc8-4cb7de5d957b None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:20.066927 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43f6c3-ead1-436b-a8b1-ea9b6a1cf30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:20.067823 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43f6c3-ead1-436b-a8b1-ea9b6a1cf30f None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:20.068111 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43f6c3-ead1-436b-a8b1-ea9b6a1cf30f None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:20.068428 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43f6c3-ead1-436b-a8b1-ea9b6a1cf30f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:20.079320 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed43f6c3-ead1-436b-a8b1-ea9b6a1cf30f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:20.081246 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed43f6c3-ead1-436b-a8b1-ea9b6a1cf30f None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:20.101530 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-ed43f6c3-ead1-436b-a8b1-ea9b6a1cf30f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:20.101530 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:20.101530 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:20.101530 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:20.101530 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:20.101530 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:20.101530 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:20.101530 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:20.101530 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:20.101530 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:20.101530 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:20.101530 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:20.101530 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 23:02:20.101530 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 23:02:20.101530 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 23:02:20.102492 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 23:02:20.102492 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:20.102492 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:20.102492 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:20.102492 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:20.102492 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:20.102492 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:20.102492 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:20.102492 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:20.102492 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:20.102492 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:20.102492 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:20.102492 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:20.102492 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:20.102492 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:20.102492 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:20.102492 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:20.103369 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 23:02:20.103369 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 23:02:20.103369 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 23:02:20.103369 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 23:02:20.103369 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 23:02:20.103369 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 23:02:20.103369 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:20.103369 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:20.107640 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 377/762] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:02:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:20.108897 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43f6c3-ead1-436b-a8b1-ea9b6a1cf30f None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:20.123667 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26346739-c65e-4a49-be93-706d8739f393 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:20.172860 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124367ec-a9df-4394-82d0-042b9f8ec5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:20.174518 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124367ec-a9df-4394-82d0-042b9f8ec5c5 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:20.175520 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124367ec-a9df-4394-82d0-042b9f8ec5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:20.176274 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124367ec-a9df-4394-82d0-042b9f8ec5c5 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:20.177424 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124367ec-a9df-4394-82d0-042b9f8ec5c5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:20.178329 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124367ec-a9df-4394-82d0-042b9f8ec5c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:20.180945 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124367ec-a9df-4394-82d0-042b9f8ec5c5 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:20.189057 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 386/763] 10.222.0.60 () {56 vars in 1124 bytes} [Wed Jul 13 23:02:19 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:20.190327 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124367ec-a9df-4394-82d0-042b9f8ec5c5 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:20.254332 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:20.260109 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43f6c3-ead1-436b-a8b1-ea9b6a1cf30f None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:20.265109 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:20.317203 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4268073-15a5-4bd7-9e05-d768335a2983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:20.317932 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4268073-15a5-4bd7-9e05-d768335a2983 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:20.318222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4268073-15a5-4bd7-9e05-d768335a2983 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:20.318502 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4268073-15a5-4bd7-9e05-d768335a2983 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:20.343092 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124367ec-a9df-4394-82d0-042b9f8ec5c5 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:20.396625 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e4db9-1544-4fab-a2d3-fe7127cf5f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:20.398296 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e4db9-1544-4fab-a2d3-fe7127cf5f62 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:20.398796 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e4db9-1544-4fab-a2d3-fe7127cf5f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:20.399304 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e4db9-1544-4fab-a2d3-fe7127cf5f62 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:20.400451 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e4db9-1544-4fab-a2d3-fe7127cf5f62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:20.401005 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e4db9-1544-4fab-a2d3-fe7127cf5f62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:20.401469 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4268073-15a5-4bd7-9e05-d768335a2983 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:20.402988 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4268073-15a5-4bd7-9e05-d768335a2983 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:20.403108 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e4db9-1544-4fab-a2d3-fe7127cf5f62 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:20.423632 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 387/764] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jul 13 23:02:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:20.424979 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e4db9-1544-4fab-a2d3-fe7127cf5f62 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:20.523687 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 378/765] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:02:20 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 23:02:20.524673 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4268073-15a5-4bd7-9e05-d768335a2983 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:20.528347 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:20.537274 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:20.541777 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e4db9-1544-4fab-a2d3-fe7127cf5f62 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:20.579875 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc482f-53f3-4511-bbcc-3e34cc12d070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:20.580976 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc482f-53f3-4511-bbcc-3e34cc12d070 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:20.581492 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc482f-53f3-4511-bbcc-3e34cc12d070 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:20.581972 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc482f-53f3-4511-bbcc-3e34cc12d070 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:20.582975 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc482f-53f3-4511-bbcc-3e34cc12d070 None admin] RBAC: Adding query filter params (interface=public, service_id=e323312e392c48bb9e2ab218ca1f502a, region_id=RegionOne) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:20.583514 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc482f-53f3-4511-bbcc-3e34cc12d070 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:20.585197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc482f-53f3-4511-bbcc-3e34cc12d070 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:20.593922 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 388/766] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Jul 13 23:02:20 2022] GET /identity/v3/endpoints?service_id=e323312e392c48bb9e2ab218ca1f502a&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:20.595128 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc482f-53f3-4511-bbcc-3e34cc12d070 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:20.643842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4268073-15a5-4bd7-9e05-d768335a2983 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:20.682069 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6137c787-e245-4420-893f-ee2acc920782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:20.682901 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6137c787-e245-4420-893f-ee2acc920782 None admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:20.683186 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6137c787-e245-4420-893f-ee2acc920782 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:20.683466 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6137c787-e245-4420-893f-ee2acc920782 None admin] PATH_INFO: `/v3/projects/ab0a5581e4e444c293815e1304e20c05/users/4789384d158b4995bd30e1e37e30f5f1/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:20.685579 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6137c787-e245-4420-893f-ee2acc920782 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab0a5581e4e444c293815e1304e20c05, user_id=4789384d158b4995bd30e1e37e30f5f1, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:20.687054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6137c787-e245-4420-893f-ee2acc920782 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:20.715585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 379/767] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jul 13 23:02:20 2022] PUT /identity/v3/projects/ab0a5581e4e444c293815e1304e20c05/users/4789384d158b4995bd30e1e37e30f5f1/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:02:20.715585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6137c787-e245-4420-893f-ee2acc920782 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:20.736447 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc482f-53f3-4511-bbcc-3e34cc12d070 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:20.786200 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:20.821332 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf4c6d-7425-4a00-b7c0-c5b995f06a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:20.822245 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf4c6d-7425-4a00-b7c0-c5b995f06a03 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:20.823089 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf4c6d-7425-4a00-b7c0-c5b995f06a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:20.823089 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf4c6d-7425-4a00-b7c0-c5b995f06a03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:20.837526 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf4c6d-7425-4a00-b7c0-c5b995f06a03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:20.839285 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf4c6d-7425-4a00-b7c0-c5b995f06a03 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:20.840479 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6137c787-e245-4420-893f-ee2acc920782 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:20.861880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-33cf4c6d-7425-4a00-b7c0-c5b995f06a03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:20.861880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:20.861880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:20.861880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:20.861880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:20.861880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:20.861880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:20.861880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:20.861880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:20.861880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:20.861880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:20.861880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:20.861880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:20.861880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:20.861880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:20.862968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:20.862968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:20.862968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:20.862968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:20.862968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:20.862968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:20.862968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:20.862968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:20.862968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:20.862968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:20.862968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:20.862968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:20.862968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:20.862968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:20.862968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:20.862968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:20.862968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:20.863911 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:20.863911 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:20.863911 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:20.863911 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:20.868060 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 389/768] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:20.869601 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0cb96-f50b-419d-8ee7-efff224d24ff None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:20.869908 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0cb96-f50b-419d-8ee7-efff224d24ff None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:20.870237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0cb96-f50b-419d-8ee7-efff224d24ff None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:20.875980 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 390/769] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:20.879331 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4512901-2675-4d19-aff5-870c4b9f51e5 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:20.879567 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4512901-2675-4d19-aff5-870c4b9f51e5 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:20.879891 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4512901-2675-4d19-aff5-870c4b9f51e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:20.890784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd449083-054a-46fd-bd5b-28ebd9515026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:20.891849 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd449083-054a-46fd-bd5b-28ebd9515026 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:20.892348 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd449083-054a-46fd-bd5b-28ebd9515026 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:20.892807 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd449083-054a-46fd-bd5b-28ebd9515026 None admin] PATH_INFO: `/v3/services` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:20.893797 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd449083-054a-46fd-bd5b-28ebd9515026 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:20.895548 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd449083-054a-46fd-bd5b-28ebd9515026 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:20.900139 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-e4512901-2675-4d19-aff5-870c4b9f51e5 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:20.901377 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e4512901-2675-4d19-aff5-870c4b9f51e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:20.911977 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 380/770] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Jul 13 23:02:20 2022] GET /identity/v3/services => generated 772 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:20.913956 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd449083-054a-46fd-bd5b-28ebd9515026 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:20.917241 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:21.009440 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 391/771] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:02:21.035332 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4512901-2675-4d19-aff5-870c4b9f51e5 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:21.044387 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd449083-054a-46fd-bd5b-28ebd9515026 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:21.078033 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:21.094159 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494737d7-7777-4ebd-bf24-87dbead5467d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:21.095405 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494737d7-7777-4ebd-bf24-87dbead5467d None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:21.095861 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494737d7-7777-4ebd-bf24-87dbead5467d None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:21.096573 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494737d7-7777-4ebd-bf24-87dbead5467d None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:21.098057 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494737d7-7777-4ebd-bf24-87dbead5467d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:21.098748 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494737d7-7777-4ebd-bf24-87dbead5467d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:21.100510 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494737d7-7777-4ebd-bf24-87dbead5467d None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:21.117739 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 381/772] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:21.123556 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494737d7-7777-4ebd-bf24-87dbead5467d None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:21.193062 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4512901-2675-4d19-aff5-870c4b9f51e5 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:21.252251 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce433c-98bf-41b9-8902-7471d8d4150d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:21.252251 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce433c-98bf-41b9-8902-7471d8d4150d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:21.252251 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce433c-98bf-41b9-8902-7471d8d4150d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:21.252251 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce433c-98bf-41b9-8902-7471d8d4150d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:21.264727 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494737d7-7777-4ebd-bf24-87dbead5467d None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:21.280403 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbce433c-98bf-41b9-8902-7471d8d4150d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:02:21.290805 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-cbce433c-98bf-41b9-8902-7471d8d4150d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:02:21.290805 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:21.290805 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:21.290805 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:21.290805 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:21.290805 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:21.290805 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:21.290805 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:21.290805 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:21.290805 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:21.290805 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:21.290805 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:21.290805 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:02:21.290805 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:02:21.290805 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:02:21.290805 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:02:21.291864 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:02:21.291864 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:02:21.291864 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:02:21.291864 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:02:21.291864 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:21.291864 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:21.291864 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:21.291864 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:21.291864 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:21.291864 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:21.291864 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:21.291864 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:21.291864 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:21.291864 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:21.291864 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:21.291864 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:21.292726 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:21.292726 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:21.292726 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:21.292726 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:21.292726 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:02:21.292726 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:02:21.292726 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:02:21.292726 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:02:21.292726 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:02:21.292726 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:02:21.292726 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:02:21.292726 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:21.295995 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 392/773] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jul 13 23:02:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:21.302677 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce433c-98bf-41b9-8902-7471d8d4150d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:21.315174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d150521-84cf-485f-9ad7-90477ad77ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:21.315577 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:21.316023 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d150521-84cf-485f-9ad7-90477ad77ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:21.316646 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d150521-84cf-485f-9ad7-90477ad77ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:21.316646 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d150521-84cf-485f-9ad7-90477ad77ca8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:21.333378 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d150521-84cf-485f-9ad7-90477ad77ca8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:21.337424 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d150521-84cf-485f-9ad7-90477ad77ca8 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:21.360907 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-4d150521-84cf-485f-9ad7-90477ad77ca8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:21.360907 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:21.360907 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:21.360907 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:21.360907 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:21.360907 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:21.360907 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:21.360907 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:21.360907 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:21.360907 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:21.360907 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:21.360907 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:21.360907 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 23:02:21.360907 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 23:02:21.360907 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 23:02:21.362174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 23:02:21.362174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:21.362174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:21.362174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:21.362174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:21.362174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:21.362174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:21.362174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:21.362174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:21.362174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:21.362174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:21.362174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:21.362174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:21.362174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:21.362174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:21.362174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:21.362174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:21.363415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 23:02:21.363415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 23:02:21.363415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 23:02:21.363415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 23:02:21.363415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 23:02:21.363415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 23:02:21.363415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:21.363415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:21.367212 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 382/774] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:02:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:21.375584 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d150521-84cf-485f-9ad7-90477ad77ca8 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:21.429206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce433c-98bf-41b9-8902-7471d8d4150d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:21.463175 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:21.470303 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df947b67-3a06-4267-a7d0-f85e5fff7899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:21.471559 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df947b67-3a06-4267-a7d0-f85e5fff7899 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:21.472308 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df947b67-3a06-4267-a7d0-f85e5fff7899 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:21.472836 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df947b67-3a06-4267-a7d0-f85e5fff7899 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:21.474076 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df947b67-3a06-4267-a7d0-f85e5fff7899 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:21.474601 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df947b67-3a06-4267-a7d0-f85e5fff7899 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:21.476707 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df947b67-3a06-4267-a7d0-f85e5fff7899 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:21.488676 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 393/775] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:02:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:02:21.505999 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df947b67-3a06-4267-a7d0-f85e5fff7899 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:21.531647 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d150521-84cf-485f-9ad7-90477ad77ca8 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:21.573045 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64936b4-12cd-4d10-bfa6-960313cbc2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:21.573323 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64936b4-12cd-4d10-bfa6-960313cbc2d2 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:21.573695 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64936b4-12cd-4d10-bfa6-960313cbc2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:21.573850 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64936b4-12cd-4d10-bfa6-960313cbc2d2 None admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:21.574650 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64936b4-12cd-4d10-bfa6-960313cbc2d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:21.576701 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64936b4-12cd-4d10-bfa6-960313cbc2d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:21.577961 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64936b4-12cd-4d10-bfa6-960313cbc2d2 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:21.602771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 383/776] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jul 13 23:02:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:02:21.610362 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64936b4-12cd-4d10-bfa6-960313cbc2d2 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:21.663276 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:21.730821 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df947b67-3a06-4267-a7d0-f85e5fff7899 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:21.816965 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67153fac-ceb9-4452-8f28-67c4309fb206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:21.818291 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67153fac-ceb9-4452-8f28-67c4309fb206 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:21.818771 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67153fac-ceb9-4452-8f28-67c4309fb206 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:21.819291 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67153fac-ceb9-4452-8f28-67c4309fb206 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:21.831762 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64936b4-12cd-4d10-bfa6-960313cbc2d2 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:21.842666 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:21.854987 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67153fac-ceb9-4452-8f28-67c4309fb206 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:21.857864 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67153fac-ceb9-4452-8f28-67c4309fb206 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:21.875143 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6caf4db-e6bd-440c-80ba-64c32eb2e81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:21.875846 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6caf4db-e6bd-440c-80ba-64c32eb2e81d None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:21.876127 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6caf4db-e6bd-440c-80ba-64c32eb2e81d None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:21.876402 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6caf4db-e6bd-440c-80ba-64c32eb2e81d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:21.877252 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6caf4db-e6bd-440c-80ba-64c32eb2e81d None admin] RBAC: Adding query filter params (role.id=92417b14d4984ed9956dcd14f97564f3, scope.project.id=ab0a5581e4e444c293815e1304e20c05, user.id=a32702a67a7b49ffa153fc59251fd169) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:21.877560 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6caf4db-e6bd-440c-80ba-64c32eb2e81d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:21.879051 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6caf4db-e6bd-440c-80ba-64c32eb2e81d None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:21.885895 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 384/777] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:02:21 2022] GET /identity/v3/role_assignments?user.id=a32702a67a7b49ffa153fc59251fd169&scope.project.id=ab0a5581e4e444c293815e1304e20c05&role.id=92417b14d4984ed9956dcd14f97564f3 => generated 259 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:02:21.899574 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-67153fac-ceb9-4452-8f28-67c4309fb206 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:21.899574 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:21.899574 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:21.899574 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:21.899574 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:21.899574 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:21.899574 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:21.899574 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:21.899574 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:21.899574 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:21.899574 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:21.899574 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:21.899574 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:21.899574 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:21.899574 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:21.900569 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:21.900569 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:21.900569 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:21.900569 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:21.900569 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:21.900569 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:21.900569 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:21.900569 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:21.900569 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:21.900569 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:21.900569 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:21.900569 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:21.900569 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:21.900569 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:21.900569 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:21.900569 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:21.900569 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:21.901537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:21.901537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:21.901537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:21.901537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:21.904190 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 394/778] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 403 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:21.909862 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6caf4db-e6bd-440c-80ba-64c32eb2e81d None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:22.067639 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6caf4db-e6bd-440c-80ba-64c32eb2e81d None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:22.105624 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f36d781-b38e-4bed-9bfb-64b3c9e87c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:22.106294 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36d781-b38e-4bed-9bfb-64b3c9e87c1e None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:22.106574 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36d781-b38e-4bed-9bfb-64b3c9e87c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:22.106843 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36d781-b38e-4bed-9bfb-64b3c9e87c1e None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:22.107585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f36d781-b38e-4bed-9bfb-64b3c9e87c1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:22.107896 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f36d781-b38e-4bed-9bfb-64b3c9e87c1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:22.109287 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f36d781-b38e-4bed-9bfb-64b3c9e87c1e None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:22.123923 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 385/779] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:22.129681 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67153fac-ceb9-4452-8f28-67c4309fb206 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:22.211850 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a6ed8-405e-4bfe-9c35-dc9c7792dead None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:22.212302 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a6ed8-405e-4bfe-9c35-dc9c7792dead None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:22.212500 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a6ed8-405e-4bfe-9c35-dc9c7792dead None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:22.214408 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 386/780] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:22.219795 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76594095-6011-44db-96fa-97bdf015403c None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:22.220085 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76594095-6011-44db-96fa-97bdf015403c None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:22.220346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76594095-6011-44db-96fa-97bdf015403c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:22.239207 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-76594095-6011-44db-96fa-97bdf015403c None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:22.247695 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-76594095-6011-44db-96fa-97bdf015403c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:22.270997 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:22.277926 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67153fac-ceb9-4452-8f28-67c4309fb206 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:22.310671 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:22.318575 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 387/781] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:02:22.343095 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2455b86-8f86-4e55-bf23-2d11a873d4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:22.344234 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2455b86-8f86-4e55-bf23-2d11a873d4c8 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:22.344659 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2455b86-8f86-4e55-bf23-2d11a873d4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:22.345071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2455b86-8f86-4e55-bf23-2d11a873d4c8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:22.360531 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2455b86-8f86-4e55-bf23-2d11a873d4c8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:22.362453 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2455b86-8f86-4e55-bf23-2d11a873d4c8 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:22.383388 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-b2455b86-8f86-4e55-bf23-2d11a873d4c8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 13 23:02:22.383388 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:22.383388 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:22.383388 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:22.383388 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:22.383388 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:22.383388 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:22.383388 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:22.383388 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:22.383388 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:22.383388 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:22.383388 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:22.383388 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 23:02:22.383388 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 23:02:22.383388 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 23:02:22.383388 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 23:02:22.384626 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:22.384626 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:22.384626 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 23:02:22.384626 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:22.384626 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 23:02:22.384626 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:22.384626 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:22.384626 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:22.384626 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:22.384626 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:22.384626 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:22.384626 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:22.384626 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:22.384626 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:22.384626 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:22.384626 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:22.384626 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:22.384626 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:22.385880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:22.385880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:22.385880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 23:02:22.385880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 23:02:22.385880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 23:02:22.385880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 23:02:22.385880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 23:02:22.385880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 23:02:22.385880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 13 23:02:22.385880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:22.391575 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 395/782] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jul 13 23:02:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:22.391775 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76594095-6011-44db-96fa-97bdf015403c None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:22.398717 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2455b86-8f86-4e55-bf23-2d11a873d4c8 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:22.522968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2455b86-8f86-4e55-bf23-2d11a873d4c8 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:22.561718 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a637f0d8-8e7b-4558-99d2-eca81f398924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:22.562408 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637f0d8-8e7b-4558-99d2-eca81f398924 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:22.562685 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637f0d8-8e7b-4558-99d2-eca81f398924 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:22.562954 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637f0d8-8e7b-4558-99d2-eca81f398924 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:22.563939 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a637f0d8-8e7b-4558-99d2-eca81f398924 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:22.564245 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a637f0d8-8e7b-4558-99d2-eca81f398924 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:22.565821 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a637f0d8-8e7b-4558-99d2-eca81f398924 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:22.587764 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 396/783] 10.222.0.60 () {56 vars in 1135 bytes} [Wed Jul 13 23:02:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:02:22.599138 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76594095-6011-44db-96fa-97bdf015403c None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:22.599597 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a637f0d8-8e7b-4558-99d2-eca81f398924 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:22.651045 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:22.653246 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5144f3aa-fe48-4cae-8962-06be353c5438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:22.654329 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5144f3aa-fe48-4cae-8962-06be353c5438 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:22.654789 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5144f3aa-fe48-4cae-8962-06be353c5438 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:22.655246 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5144f3aa-fe48-4cae-8962-06be353c5438 None admin] PATH_INFO: `/v3/services/network` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:22.656327 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5144f3aa-fe48-4cae-8962-06be353c5438 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:22.658712 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5144f3aa-fe48-4cae-8962-06be353c5438 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:22.676599 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-5144f3aa-fe48-4cae-8962-06be353c5438 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 23:02:22.676599 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:22.676599 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 23:02:22.676599 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 23:02:22.676599 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 23:02:22.676599 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 23:02:22.676599 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 23:02:22.676599 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:22.676599 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 23:02:22.676599 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:22.676599 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 23:02:22.676599 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:22.676599 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:22.676599 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:22.676599 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:22.676599 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:22.676599 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:22.679101 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:22.679101 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:22.679101 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:22.679101 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:22.679101 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:22.679101 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:22.679101 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 23:02:22.679101 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 23:02:22.679101 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 23:02:22.679101 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 23:02:22.679101 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:22.679101 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:22.679101 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:22.679101 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:22.679101 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:22.679101 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:22.680976 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:22.680976 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:22.680976 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:22.680976 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:22.680976 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:22.680976 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:22.680976 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:22.680976 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:22.680976 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:22.680976 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:22.680976 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 23:02:22.680976 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:22.680976 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 23:02:22.680976 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:22.680976 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 388/784] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:02:22 2022] GET /identity/v3/services/network => generated 88 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:22.689422 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5144f3aa-fe48-4cae-8962-06be353c5438 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:22.758698 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a637f0d8-8e7b-4558-99d2-eca81f398924 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:22.764020 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:22.799695 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96674e5c-064f-482e-964c-dcbaf01a36b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:22.800526 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96674e5c-064f-482e-964c-dcbaf01a36b2 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:22.800813 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96674e5c-064f-482e-964c-dcbaf01a36b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:22.801089 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96674e5c-064f-482e-964c-dcbaf01a36b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:22.813499 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96674e5c-064f-482e-964c-dcbaf01a36b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:22.814988 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96674e5c-064f-482e-964c-dcbaf01a36b2 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:22.836230 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-96674e5c-064f-482e-964c-dcbaf01a36b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:22.836230 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:22.836230 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:22.836230 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:22.836230 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:22.836230 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:22.836230 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:22.836230 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:22.836230 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:22.836230 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:22.836230 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:22.836230 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:22.836230 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:22.836230 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:22.836230 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:22.838500 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:22.838500 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:22.838500 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:22.838500 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:22.838500 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:22.838500 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:22.838500 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:22.838500 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:22.838500 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:22.838500 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:22.838500 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:22.838500 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:22.838500 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:22.838500 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:22.838500 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:22.838500 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:22.838500 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:22.840772 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:22.840772 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:22.840772 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:22.840772 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:22.840772 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 397/785] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:22.849609 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96674e5c-064f-482e-964c-dcbaf01a36b2 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:22.973569 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5144f3aa-fe48-4cae-8962-06be353c5438 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:22.990287 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:23.003622 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96674e5c-064f-482e-964c-dcbaf01a36b2 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:23.042650 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5011f683-0624-4db8-b1e0-d99330052e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:23.043898 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011f683-0624-4db8-b1e0-d99330052e71 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:23.044371 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011f683-0624-4db8-b1e0-d99330052e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:23.044857 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011f683-0624-4db8-b1e0-d99330052e71 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:23.045915 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5011f683-0624-4db8-b1e0-d99330052e71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:23.046428 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5011f683-0624-4db8-b1e0-d99330052e71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:23.048365 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5011f683-0624-4db8-b1e0-d99330052e71 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:23.062108 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 398/786] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:23.067934 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5011f683-0624-4db8-b1e0-d99330052e71 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:23.084283 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30fd6c9-e9ae-4656-8c1e-a3fbddbc28c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:23.085146 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30fd6c9-e9ae-4656-8c1e-a3fbddbc28c7 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:23.085583 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30fd6c9-e9ae-4656-8c1e-a3fbddbc28c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:23.085821 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30fd6c9-e9ae-4656-8c1e-a3fbddbc28c7 None admin] PATH_INFO: `/v3/services` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:23.086690 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30fd6c9-e9ae-4656-8c1e-a3fbddbc28c7 None admin] RBAC: Adding query filter params (name=network) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:23.086978 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30fd6c9-e9ae-4656-8c1e-a3fbddbc28c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:23.088561 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30fd6c9-e9ae-4656-8c1e-a3fbddbc28c7 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:23.105942 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 389/787] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Jul 13 23:02:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:23.110964 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30fd6c9-e9ae-4656-8c1e-a3fbddbc28c7 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:23.182614 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5011f683-0624-4db8-b1e0-d99330052e71 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:23.218881 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b7f93-86fd-4bcb-9cc7-cf6116f6fa30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:23.220084 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b7f93-86fd-4bcb-9cc7-cf6116f6fa30 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:23.220546 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b7f93-86fd-4bcb-9cc7-cf6116f6fa30 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:23.221207 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b7f93-86fd-4bcb-9cc7-cf6116f6fa30 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:23.230031 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2b7f93-86fd-4bcb-9cc7-cf6116f6fa30 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:23.231759 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2b7f93-86fd-4bcb-9cc7-cf6116f6fa30 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:23.247262 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-3a2b7f93-86fd-4bcb-9cc7-cf6116f6fa30 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:23.247262 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:23.247262 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:23.247262 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:23.247262 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:23.247262 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:23.247262 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:23.247262 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:23.247262 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:23.247262 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:23.247262 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:23.247262 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:23.247262 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 23:02:23.247262 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 23:02:23.247262 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 23:02:23.248217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 23:02:23.248217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:23.248217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:23.248217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:23.248217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:23.248217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:23.248217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:23.248217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:23.248217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:23.248217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:23.248217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:23.248217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:23.248217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:23.248217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:23.248217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:23.248217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:23.248217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:23.249107 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 23:02:23.249107 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 23:02:23.249107 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 23:02:23.249107 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 23:02:23.249107 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 23:02:23.249107 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 23:02:23.249107 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:23.249107 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:23.255718 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 399/788] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:02:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:23.257279 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:23.258841 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b7f93-86fd-4bcb-9cc7-cf6116f6fa30 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:23.267314 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30fd6c9-e9ae-4656-8c1e-a3fbddbc28c7 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:23.273139 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:23.304571 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e964b-1b6d-4150-8456-c9829562333d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:23.305247 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e964b-1b6d-4150-8456-c9829562333d None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:23.305527 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e964b-1b6d-4150-8456-c9829562333d None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:23.305801 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e964b-1b6d-4150-8456-c9829562333d None admin] PATH_INFO: `/v3/services` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:23.306516 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5e964b-1b6d-4150-8456-c9829562333d None admin] RBAC: Adding query filter params (type=network) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:23.306819 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5e964b-1b6d-4150-8456-c9829562333d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:23.308231 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5e964b-1b6d-4150-8456-c9829562333d None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:23.315039 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 390/789] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Jul 13 23:02:23 2022] GET /identity/v3/services?type=network => generated 350 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:23.320669 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e964b-1b6d-4150-8456-c9829562333d None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:23.379956 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b7f93-86fd-4bcb-9cc7-cf6116f6fa30 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:23.419111 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b504fbd2-1a65-4f6e-a025-1d9cae3d075e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:23.420019 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504fbd2-1a65-4f6e-a025-1d9cae3d075e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:23.420370 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504fbd2-1a65-4f6e-a025-1d9cae3d075e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:23.420807 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504fbd2-1a65-4f6e-a025-1d9cae3d075e None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:23.421673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b504fbd2-1a65-4f6e-a025-1d9cae3d075e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:23.422045 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b504fbd2-1a65-4f6e-a025-1d9cae3d075e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:23.423757 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b504fbd2-1a65-4f6e-a025-1d9cae3d075e None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:23.426552 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e964b-1b6d-4150-8456-c9829562333d None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:23.437965 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 400/790] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jul 13 23:02:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:02:23.443724 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b504fbd2-1a65-4f6e-a025-1d9cae3d075e None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:23.462550 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc688cbc-58b1-4e1b-af29-9542af6f0ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:23.463217 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc688cbc-58b1-4e1b-af29-9542af6f0ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:23.463498 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc688cbc-58b1-4e1b-af29-9542af6f0ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:23.463805 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc688cbc-58b1-4e1b-af29-9542af6f0ee9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:23.464605 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc688cbc-58b1-4e1b-af29-9542af6f0ee9 None admin] RBAC: Adding query filter params (interface=public, service_id=d0dfc9f1ca54416cb08d9f1e6c2c6241, region_id=RegionOne) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:23.464913 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc688cbc-58b1-4e1b-af29-9542af6f0ee9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:23.466297 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc688cbc-58b1-4e1b-af29-9542af6f0ee9 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:23.474432 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 391/791] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Jul 13 23:02:23 2022] GET /identity/v3/endpoints?service_id=d0dfc9f1ca54416cb08d9f1e6c2c6241&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:02:23.480354 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc688cbc-58b1-4e1b-af29-9542af6f0ee9 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:23.500815 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:23.524317 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:23.560758 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b504fbd2-1a65-4f6e-a025-1d9cae3d075e None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:23.585735 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc688cbc-58b1-4e1b-af29-9542af6f0ee9 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:23.598544 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83010ed-3ec0-41e5-87fa-9632ad267e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:23.599563 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83010ed-3ec0-41e5-87fa-9632ad267e22 None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:23.599939 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83010ed-3ec0-41e5-87fa-9632ad267e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:23.600756 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83010ed-3ec0-41e5-87fa-9632ad267e22 None admin] PATH_INFO: `/v3/projects/ab0a5581e4e444c293815e1304e20c05/users/786fc44611804d3ca6280ffcab415569/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:23.602887 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83010ed-3ec0-41e5-87fa-9632ad267e22 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab0a5581e4e444c293815e1304e20c05, user_id=786fc44611804d3ca6280ffcab415569, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:23.604589 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83010ed-3ec0-41e5-87fa-9632ad267e22 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:23.621212 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 401/792] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jul 13 23:02:23 2022] PUT /identity/v3/projects/ab0a5581e4e444c293815e1304e20c05/users/786fc44611804d3ca6280ffcab415569/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:02:23.621505 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c448d-9ba4-43c0-b1bd-f481197d7598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:23.622232 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c448d-9ba4-43c0-b1bd-f481197d7598 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:23.622525 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c448d-9ba4-43c0-b1bd-f481197d7598 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:23.622804 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c448d-9ba4-43c0-b1bd-f481197d7598 None admin] PATH_INFO: `/v3/services` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:23.623585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c448d-9ba4-43c0-b1bd-f481197d7598 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:23.625024 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c448d-9ba4-43c0-b1bd-f481197d7598 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:23.627893 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1a537-1706-4bb2-b86d-a02eea7d05b1 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:23.628325 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1a537-1706-4bb2-b86d-a02eea7d05b1 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:23.628609 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1a537-1706-4bb2-b86d-a02eea7d05b1 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:23.630426 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 402/793] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:23.631375 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 392/794] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Jul 13 23:02:23 2022] GET /identity/v3/services => generated 772 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:23.636724 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e771eb31-a764-4073-a3c9-b236bf292778 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:23.637068 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e771eb31-a764-4073-a3c9-b236bf292778 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:23.637355 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e771eb31-a764-4073-a3c9-b236bf292778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:23.666427 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-e771eb31-a764-4073-a3c9-b236bf292778 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:23.667810 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e771eb31-a764-4073-a3c9-b236bf292778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:23.795229 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 403/795] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:02:23.828908 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c448d-9ba4-43c0-b1bd-f481197d7598 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:23.848941 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3966b7-2400-49de-a26e-439608de6e6e None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:23.849282 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3966b7-2400-49de-a26e-439608de6e6e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:23.849486 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3966b7-2400-49de-a26e-439608de6e6e None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:23.851778 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 404/796] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:23.857381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af283267-f209-4597-89e6-cff7968390a8 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:23.857686 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af283267-f209-4597-89e6-cff7968390a8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:23.857919 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af283267-f209-4597-89e6-cff7968390a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:23.882382 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-af283267-f209-4597-89e6-cff7968390a8 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:23.883659 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-af283267-f209-4597-89e6-cff7968390a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:23.935299 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:23.955648 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c448d-9ba4-43c0-b1bd-f481197d7598 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:23.967497 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:23.986128 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 405/797] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 23:02:23.992394 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a93184-38fb-4fc0-ac68-bba7a89e482b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:23.993254 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a93184-38fb-4fc0-ac68-bba7a89e482b None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:23.993536 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a93184-38fb-4fc0-ac68-bba7a89e482b None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:23.993814 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a93184-38fb-4fc0-ac68-bba7a89e482b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:24.005115 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a93184-38fb-4fc0-ac68-bba7a89e482b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:02:24.013608 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-66a93184-38fb-4fc0-ac68-bba7a89e482b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:02:24.013608 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:24.013608 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:24.013608 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:24.013608 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:24.013608 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:24.013608 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:24.013608 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:24.013608 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:24.013608 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:24.013608 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:24.013608 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:24.013608 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:02:24.013608 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:02:24.013608 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:02:24.013608 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:02:24.014702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:02:24.014702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:02:24.014702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:02:24.014702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:02:24.014702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:24.014702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:24.014702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:24.014702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:24.014702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:24.014702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:24.014702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:24.014702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:24.014702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:24.014702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:24.014702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:24.014702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:24.015729 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:24.015729 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:24.015729 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:24.015729 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:24.015729 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:02:24.015729 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:02:24.015729 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:02:24.015729 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:02:24.015729 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:02:24.015729 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:02:24.015729 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:02:24.015729 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:24.015729 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 393/798] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jul 13 23:02:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:24.022651 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af283267-f209-4597-89e6-cff7968390a8 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:24.023523 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a93184-38fb-4fc0-ac68-bba7a89e482b None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:24.264886 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af283267-f209-4597-89e6-cff7968390a8 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:24.291561 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a93184-38fb-4fc0-ac68-bba7a89e482b None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:24.313681 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf51b4db-a724-41b5-afb1-57b577e56d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:24.314619 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51b4db-a724-41b5-afb1-57b577e56d62 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:24.315045 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51b4db-a724-41b5-afb1-57b577e56d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:24.315480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51b4db-a724-41b5-afb1-57b577e56d62 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:24.316397 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf51b4db-a724-41b5-afb1-57b577e56d62 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:24.316857 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf51b4db-a724-41b5-afb1-57b577e56d62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:24.318689 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf51b4db-a724-41b5-afb1-57b577e56d62 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:24.337000 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 406/799] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:02:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:24.342748 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf51b4db-a724-41b5-afb1-57b577e56d62 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:24.374173 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81ff32-11d8-4490-b06f-e8cfdc55a999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:24.375514 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81ff32-11d8-4490-b06f-e8cfdc55a999 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:24.375984 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81ff32-11d8-4490-b06f-e8cfdc55a999 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:24.376600 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81ff32-11d8-4490-b06f-e8cfdc55a999 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:24.377839 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81ff32-11d8-4490-b06f-e8cfdc55a999 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:24.379424 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81ff32-11d8-4490-b06f-e8cfdc55a999 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:24.394882 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-ad81ff32-11d8-4490-b06f-e8cfdc55a999 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 23:02:24.394882 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:24.394882 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 23:02:24.394882 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 23:02:24.394882 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 23:02:24.394882 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 23:02:24.394882 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 23:02:24.394882 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:24.394882 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 23:02:24.394882 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:24.394882 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 23:02:24.394882 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:24.394882 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:24.394882 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:24.394882 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:24.394882 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:24.396100 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:24.396100 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:24.396100 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:24.396100 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:24.396100 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:24.396100 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:24.396100 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:24.396100 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 23:02:24.396100 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 23:02:24.396100 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 23:02:24.396100 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 23:02:24.396100 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:24.396100 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:24.396100 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:24.396100 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:24.396100 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:24.397116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:24.397116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:24.397116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:24.397116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:24.397116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:24.397116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:24.397116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:24.397116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:24.397116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:24.397116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:24.397116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:24.397116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 23:02:24.397116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:24.397116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 23:02:24.397116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:24.400641 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 394/800] 10.222.0.60 () {56 vars in 1129 bytes} [Wed Jul 13 23:02:24 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:24.409284 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81ff32-11d8-4490-b06f-e8cfdc55a999 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:24.435929 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:24.463150 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:24.536760 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf51b4db-a724-41b5-afb1-57b577e56d62 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:24.586420 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81ff32-11d8-4490-b06f-e8cfdc55a999 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:24.592764 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e6244-0b28-4935-af5f-e0c27af2f6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:24.593570 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e6244-0b28-4935-af5f-e0c27af2f6d4 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:24.593852 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e6244-0b28-4935-af5f-e0c27af2f6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:24.594676 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e6244-0b28-4935-af5f-e0c27af2f6d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:24.610040 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e6244-0b28-4935-af5f-e0c27af2f6d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:24.612556 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e6244-0b28-4935-af5f-e0c27af2f6d4 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:24.622696 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694dd676-93c5-450f-81b6-4ea0dd81bd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:24.623372 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694dd676-93c5-450f-81b6-4ea0dd81bd2c None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:24.623684 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694dd676-93c5-450f-81b6-4ea0dd81bd2c None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:24.623959 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694dd676-93c5-450f-81b6-4ea0dd81bd2c None admin] PATH_INFO: `/v3/services` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:24.624668 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694dd676-93c5-450f-81b6-4ea0dd81bd2c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:24.624973 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694dd676-93c5-450f-81b6-4ea0dd81bd2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:24.626746 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694dd676-93c5-450f-81b6-4ea0dd81bd2c None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:24.633789 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 395/801] 10.222.0.60 () {56 vars in 1124 bytes} [Wed Jul 13 23:02:24 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:24.635515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8f5af-4c6e-407c-bfd4-116b07b52e9b None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:24.635926 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8f5af-4c6e-407c-bfd4-116b07b52e9b None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:24.636212 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8f5af-4c6e-407c-bfd4-116b07b52e9b None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:24.642363 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-033e6244-0b28-4935-af5f-e0c27af2f6d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:24.642363 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:24.642363 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:24.642363 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:24.642363 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:24.642363 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:24.642363 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:24.642363 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:24.642363 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:24.642363 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:24.642363 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:24.642363 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:24.642363 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:24.642363 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:24.642363 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:24.643831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:24.643831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:24.643831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:24.643831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:24.643831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:24.643831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:24.643831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:24.643831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:24.643831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:24.643831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:24.643831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:24.643831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:24.643831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:24.643831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:24.643831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:24.643831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:24.643831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:24.644787 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:24.644787 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:24.644787 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:24.644787 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:24.645080 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 396/802] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:24.645080 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4a400-6fea-4197-9191-c12500298c71 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:24.645215 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4a400-6fea-4197-9191-c12500298c71 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:24.645656 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4a400-6fea-4197-9191-c12500298c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:24.645831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 407/803] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:02:24.651558 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e6244-0b28-4935-af5f-e0c27af2f6d4 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:24.663785 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-9eb4a400-6fea-4197-9191-c12500298c71 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:24.665014 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-9eb4a400-6fea-4197-9191-c12500298c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:24.736735 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:24.742532 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 397/804] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:02:24.748454 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb4a400-6fea-4197-9191-c12500298c71 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:24.788599 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e6244-0b28-4935-af5f-e0c27af2f6d4 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:24.791919 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:24.837515 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b267d4-aff9-4496-84dc-89a2a294b768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:24.838176 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b267d4-aff9-4496-84dc-89a2a294b768 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:24.838453 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b267d4-aff9-4496-84dc-89a2a294b768 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:24.838721 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b267d4-aff9-4496-84dc-89a2a294b768 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:24.839455 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b267d4-aff9-4496-84dc-89a2a294b768 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:24.839754 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b267d4-aff9-4496-84dc-89a2a294b768 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:24.841121 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b267d4-aff9-4496-84dc-89a2a294b768 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:24.857657 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 408/805] 10.222.0.60 () {56 vars in 1124 bytes} [Wed Jul 13 23:02:24 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:24.857657 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b267d4-aff9-4496-84dc-89a2a294b768 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:24.892240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb4a400-6fea-4197-9191-c12500298c71 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:24.939578 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbf05b9-c82d-41a9-9a59-85c7693b9ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:24.941324 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf05b9-c82d-41a9-9a59-85c7693b9ce9 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:24.941960 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf05b9-c82d-41a9-9a59-85c7693b9ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:24.942525 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf05b9-c82d-41a9-9a59-85c7693b9ce9 None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:24.943450 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbf05b9-c82d-41a9-9a59-85c7693b9ce9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:24.943995 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbf05b9-c82d-41a9-9a59-85c7693b9ce9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:24.946114 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbf05b9-c82d-41a9-9a59-85c7693b9ce9 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:24.962708 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 398/806] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:24.963760 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbf05b9-c82d-41a9-9a59-85c7693b9ce9 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:24.987635 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b267d4-aff9-4496-84dc-89a2a294b768 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:25.032552 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f705f7-f809-4af5-b61f-0f900450af48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:25.034140 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f705f7-f809-4af5-b61f-0f900450af48 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:25.034726 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f705f7-f809-4af5-b61f-0f900450af48 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:25.035148 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f705f7-f809-4af5-b61f-0f900450af48 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:25.036950 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:25.052610 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f705f7-f809-4af5-b61f-0f900450af48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:02:25.063703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-37f705f7-f809-4af5-b61f-0f900450af48 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:02:25.063703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:25.063703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:25.063703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:25.063703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:25.063703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:25.063703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:25.063703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:25.063703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:25.063703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:25.063703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:25.063703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:25.063703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:02:25.063703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:02:25.063703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:02:25.063703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:02:25.064888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:02:25.064888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:02:25.064888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:02:25.064888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:02:25.064888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:25.064888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:25.064888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:25.064888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:25.064888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:25.064888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:25.064888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:25.064888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:25.064888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:25.064888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:25.064888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:25.064888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:25.065918 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:25.065918 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:25.065918 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:25.065918 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:25.065918 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:02:25.065918 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:02:25.065918 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:02:25.065918 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:02:25.065918 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:02:25.065918 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:02:25.065918 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:02:25.065918 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:25.074023 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 409/807] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jul 13 23:02:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:25.075717 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f705f7-f809-4af5-b61f-0f900450af48 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:25.093331 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbf05b9-c82d-41a9-9a59-85c7693b9ce9 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:25.117083 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:25.131397 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d431f0ad-314a-453f-a1a5-0e6fce5d68d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:25.132257 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d431f0ad-314a-453f-a1a5-0e6fce5d68d9 None admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:25.132649 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d431f0ad-314a-453f-a1a5-0e6fce5d68d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:25.133019 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d431f0ad-314a-453f-a1a5-0e6fce5d68d9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:25.133962 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d431f0ad-314a-453f-a1a5-0e6fce5d68d9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:25.135438 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d431f0ad-314a-453f-a1a5-0e6fce5d68d9 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:25.167564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 399/808] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Jul 13 23:02:24 2022] POST /identity/v3/endpoints => generated 346 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:02:25.167564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d431f0ad-314a-453f-a1a5-0e6fce5d68d9 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:25.229701 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f705f7-f809-4af5-b61f-0f900450af48 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:25.288528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d431f0ad-314a-453f-a1a5-0e6fce5d68d9 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:25.316025 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:25.321241 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffa8b5f-02d2-43cb-b6c8-1da01c85138f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:25.322060 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa8b5f-02d2-43cb-b6c8-1da01c85138f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:25.322341 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa8b5f-02d2-43cb-b6c8-1da01c85138f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:25.322614 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa8b5f-02d2-43cb-b6c8-1da01c85138f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:25.341161 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffa8b5f-02d2-43cb-b6c8-1da01c85138f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:25.342637 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffa8b5f-02d2-43cb-b6c8-1da01c85138f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:25.347784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca9b0fe-1a86-488c-a186-7f464738b1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:25.348556 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9b0fe-1a86-488c-a186-7f464738b1ce None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:25.348840 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9b0fe-1a86-488c-a186-7f464738b1ce None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:25.349503 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9b0fe-1a86-488c-a186-7f464738b1ce None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:25.350815 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca9b0fe-1a86-488c-a186-7f464738b1ce None admin] RBAC: Adding query filter params (name=service) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:25.351144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca9b0fe-1a86-488c-a186-7f464738b1ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:25.353660 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca9b0fe-1a86-488c-a186-7f464738b1ce None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:25.388888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-8ffa8b5f-02d2-43cb-b6c8-1da01c85138f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 23:02:25.388888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:25.388888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:25.388888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:25.388888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:25.388888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:25.388888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:25.388888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:25.388888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:25.388888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:25.388888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:25.388888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:25.388888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 23:02:25.388888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 23:02:25.388888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 23:02:25.388888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 23:02:25.389890 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:25.389890 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:25.389890 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 23:02:25.389890 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:25.389890 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 23:02:25.389890 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:25.389890 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:25.389890 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:25.389890 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:25.389890 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:25.389890 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:25.389890 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:25.389890 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:25.389890 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:25.389890 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:25.389890 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:25.389890 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:25.389890 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:25.390812 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:25.390812 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:25.390812 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 23:02:25.390812 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 23:02:25.390812 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 23:02:25.390812 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 23:02:25.390812 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 23:02:25.390812 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 23:02:25.390812 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 23:02:25.390812 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:25.391855 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 400/810] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jul 13 23:02:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:25.395084 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 410/810] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Jul 13 23:02:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:25.396553 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca9b0fe-1a86-488c-a186-7f464738b1ce None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:25.397796 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffa8b5f-02d2-43cb-b6c8-1da01c85138f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:25.500136 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:25.513816 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffa8b5f-02d2-43cb-b6c8-1da01c85138f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:25.514347 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca9b0fe-1a86-488c-a186-7f464738b1ce None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:25.556553 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4cdcc-64f9-42b6-879b-a70fca882f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:25.557099 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758194b-5f7d-4f10-b6a5-ff24004bf2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:25.557604 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4cdcc-64f9-42b6-879b-a70fca882f37 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:25.557604 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4cdcc-64f9-42b6-879b-a70fca882f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:25.557894 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4cdcc-64f9-42b6-879b-a70fca882f37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:25.558741 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758194b-5f7d-4f10-b6a5-ff24004bf2a4 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:25.559235 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758194b-5f7d-4f10-b6a5-ff24004bf2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:25.559863 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758194b-5f7d-4f10-b6a5-ff24004bf2a4 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:25.561004 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758194b-5f7d-4f10-b6a5-ff24004bf2a4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:25.561524 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758194b-5f7d-4f10-b6a5-ff24004bf2a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:25.563165 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758194b-5f7d-4f10-b6a5-ff24004bf2a4 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:25.569788 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4cdcc-64f9-42b6-879b-a70fca882f37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:25.572293 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4cdcc-64f9-42b6-879b-a70fca882f37 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:25.584148 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 411/811] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Jul 13 23:02:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:25.596014 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758194b-5f7d-4f10-b6a5-ff24004bf2a4 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:25.599281 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-eae4cdcc-64f9-42b6-879b-a70fca882f37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:25.599281 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:25.599281 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:25.599281 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:25.599281 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:25.599281 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:25.599281 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:25.599281 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:25.599281 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:25.599281 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:25.599281 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:25.599281 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:25.599281 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:25.599281 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:25.599281 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:25.600734 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:25.600734 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:25.600734 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:25.600734 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:25.600734 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:25.600734 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:25.600734 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:25.600734 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:25.600734 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:25.600734 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:25.600734 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:25.600734 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:25.600734 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:25.600734 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:25.600734 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:25.600734 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:25.600734 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:25.601669 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:25.601669 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:25.601669 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:25.601669 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:25.603803 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 401/812] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:25.609674 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4cdcc-64f9-42b6-879b-a70fca882f37 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:25.639405 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:25.727555 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758194b-5f7d-4f10-b6a5-ff24004bf2a4 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:25.730973 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4cdcc-64f9-42b6-879b-a70fca882f37 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:25.802120 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01392e4-b0fc-4d1e-b44e-ecd9b2840a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:25.802777 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01392e4-b0fc-4d1e-b44e-ecd9b2840a11 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:25.803060 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01392e4-b0fc-4d1e-b44e-ecd9b2840a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:25.803924 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01392e4-b0fc-4d1e-b44e-ecd9b2840a11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:25.814680 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad6c8f-72d1-4f76-a64e-ffa3af344460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:25.815356 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad6c8f-72d1-4f76-a64e-ffa3af344460 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:25.815662 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad6c8f-72d1-4f76-a64e-ffa3af344460 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:25.815939 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad6c8f-72d1-4f76-a64e-ffa3af344460 None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:25.816673 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad6c8f-72d1-4f76-a64e-ffa3af344460 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:25.816978 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad6c8f-72d1-4f76-a64e-ffa3af344460 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:25.818370 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad6c8f-72d1-4f76-a64e-ffa3af344460 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:25.842127 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:25.854992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 402/813] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:02:25.856360 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad6c8f-72d1-4f76-a64e-ffa3af344460 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:25.956898 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:25.968827 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01392e4-b0fc-4d1e-b44e-ecd9b2840a11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:25.970891 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01392e4-b0fc-4d1e-b44e-ecd9b2840a11 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:25.974126 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad6c8f-72d1-4f76-a64e-ffa3af344460 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:26.011506 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015f390-f084-465f-8ac2-6b46450ade49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:26.012682 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015f390-f084-465f-8ac2-6b46450ade49 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:26.013118 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015f390-f084-465f-8ac2-6b46450ade49 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:26.013398 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015f390-f084-465f-8ac2-6b46450ade49 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:26.022854 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e015f390-f084-465f-8ac2-6b46450ade49 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:26.024464 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e015f390-f084-465f-8ac2-6b46450ade49 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:26.041126 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-e015f390-f084-465f-8ac2-6b46450ade49 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 23:02:26.041126 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:26.041126 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:26.041126 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:26.041126 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:26.041126 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:26.041126 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:26.041126 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:26.041126 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:26.041126 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:26.041126 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:26.041126 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:26.041126 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 23:02:26.041126 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 23:02:26.041126 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 23:02:26.041126 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 23:02:26.042558 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:26.042558 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:26.042558 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 23:02:26.042558 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:26.042558 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 23:02:26.042558 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:26.042558 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:26.042558 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:26.042558 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:26.042558 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:26.042558 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:26.042558 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:26.042558 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:26.042558 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:26.042558 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:26.042558 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:26.042558 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:26.042558 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:26.043508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:26.043508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:26.043508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 23:02:26.043508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 23:02:26.043508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 23:02:26.043508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 23:02:26.043508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 23:02:26.043508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 23:02:26.043508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 23:02:26.043508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:26.043508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 403/814] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Jul 13 23:02:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:26.044135 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59506325-b071-446c-8039-cc17ceaaf0cd None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:26.044706 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59506325-b071-446c-8039-cc17ceaaf0cd None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:26.044706 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59506325-b071-446c-8039-cc17ceaaf0cd None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:26.048651 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 404/815] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:26.050947 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ba336-3e78-4666-bde4-5c0b65f1a0ca None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:26.051234 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ba336-3e78-4666-bde4-5c0b65f1a0ca None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:26.051642 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ba336-3e78-4666-bde4-5c0b65f1a0ca None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:26.053015 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 405/816] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:26.053590 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ba336-3e78-4666-bde4-5c0b65f1a0ca None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:26.095756 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 412/817] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:02:25 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 23:02:26.096959 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da4625-eafa-45b6-a222-0679d5196ae5 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:26.097243 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da4625-eafa-45b6-a222-0679d5196ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:26.097515 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da4625-eafa-45b6-a222-0679d5196ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:26.115808 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-27da4625-eafa-45b6-a222-0679d5196ae5 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:26.117063 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-27da4625-eafa-45b6-a222-0679d5196ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:26.165425 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:26.169607 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ba336-3e78-4666-bde4-5c0b65f1a0ca None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:26.199690 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 413/818] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:02:26.201751 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd6665-f2d1-42ed-99f1-4b1baa017824 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:26.202107 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd6665-f2d1-42ed-99f1-4b1baa017824 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:26.202397 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd6665-f2d1-42ed-99f1-4b1baa017824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:26.210295 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc691e77-b2ac-4a7a-a586-bef40e8dd799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:26.210971 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc691e77-b2ac-4a7a-a586-bef40e8dd799 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:26.211298 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc691e77-b2ac-4a7a-a586-bef40e8dd799 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:26.211624 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc691e77-b2ac-4a7a-a586-bef40e8dd799 None admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:26.212474 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc691e77-b2ac-4a7a-a586-bef40e8dd799 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:26.212780 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc691e77-b2ac-4a7a-a586-bef40e8dd799 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:26.214874 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc691e77-b2ac-4a7a-a586-bef40e8dd799 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:26.220763 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-a8fd6665-f2d1-42ed-99f1-4b1baa017824 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:26.222003 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a8fd6665-f2d1-42ed-99f1-4b1baa017824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:26.235707 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 406/819] 10.222.0.60 () {56 vars in 1149 bytes} [Wed Jul 13 23:02:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:26.236548 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc691e77-b2ac-4a7a-a586-bef40e8dd799 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:26.241351 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:26.304402 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 414/820] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:02:26.305152 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd6665-f2d1-42ed-99f1-4b1baa017824 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:26.368803 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc691e77-b2ac-4a7a-a586-bef40e8dd799 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:26.408856 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca9f41-a335-4e35-853c-dfd511dd5d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:26.409645 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca9f41-a335-4e35-853c-dfd511dd5d87 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:26.409923 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca9f41-a335-4e35-853c-dfd511dd5d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:26.410200 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca9f41-a335-4e35-853c-dfd511dd5d87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:26.416605 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fd6665-f2d1-42ed-99f1-4b1baa017824 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:26.421517 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca9f41-a335-4e35-853c-dfd511dd5d87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:26.422983 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca9f41-a335-4e35-853c-dfd511dd5d87 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:26.424921 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:26.442456 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-f5ca9f41-a335-4e35-853c-dfd511dd5d87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:26.442456 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:26.442456 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:26.442456 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:26.442456 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:26.442456 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:26.442456 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:26.442456 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:26.442456 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:26.442456 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:26.442456 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:26.442456 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:26.442456 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:26.442456 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:26.442456 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:26.443415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:26.443415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:26.443415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:26.443415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:26.443415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:26.443415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:26.443415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:26.443415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:26.443415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:26.443415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:26.443415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:26.443415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:26.443415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:26.443415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:26.443415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:26.443415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:26.443415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:26.444388 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:26.444388 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:26.444388 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:26.444388 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:26.444388 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 407/821] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:26.445210 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca9f41-a335-4e35-853c-dfd511dd5d87 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:26.459040 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c4621-d58d-471b-a04a-aee86ee779ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:26.459962 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c4621-d58d-471b-a04a-aee86ee779ce None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:26.460244 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c4621-d58d-471b-a04a-aee86ee779ce None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:26.460517 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c4621-d58d-471b-a04a-aee86ee779ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:26.471600 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35c4621-d58d-471b-a04a-aee86ee779ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:02:26.479977 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-e35c4621-d58d-471b-a04a-aee86ee779ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:02:26.479977 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:26.479977 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:26.479977 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:26.479977 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:26.479977 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:26.479977 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:26.479977 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:26.479977 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:26.479977 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:26.479977 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:26.479977 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:26.479977 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:02:26.479977 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:02:26.479977 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:02:26.479977 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:02:26.480984 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:02:26.480984 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:02:26.480984 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:02:26.480984 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:02:26.480984 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:26.480984 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:26.480984 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:26.480984 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:26.480984 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:26.480984 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:26.480984 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:26.480984 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:26.480984 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:26.480984 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:26.480984 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:26.480984 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:26.481892 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:26.481892 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:26.481892 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:26.481892 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:26.481892 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:02:26.481892 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:02:26.481892 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:02:26.481892 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:02:26.481892 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:02:26.481892 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:02:26.481892 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:02:26.481892 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:26.481892 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 415/822] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jul 13 23:02:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:26.490672 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c4621-d58d-471b-a04a-aee86ee779ce None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:26.515330 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:26.562386 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca9f41-a335-4e35-853c-dfd511dd5d87 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:26.601902 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91036b62-2087-4cff-99d8-94e83b5ed4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:26.602706 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91036b62-2087-4cff-99d8-94e83b5ed4a1 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:26.602988 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91036b62-2087-4cff-99d8-94e83b5ed4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:26.603327 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91036b62-2087-4cff-99d8-94e83b5ed4a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:26.611307 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c4621-d58d-471b-a04a-aee86ee779ce None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:26.614984 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91036b62-2087-4cff-99d8-94e83b5ed4a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:26.617154 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91036b62-2087-4cff-99d8-94e83b5ed4a1 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:26.638079 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-91036b62-2087-4cff-99d8-94e83b5ed4a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:26.638079 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:26.638079 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:26.638079 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:26.638079 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:26.638079 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:26.638079 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:26.638079 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:26.638079 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:26.638079 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:26.638079 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:26.638079 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:26.638079 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:26.638079 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:26.638079 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:26.638992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:26.638992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:26.638992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:26.638992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:26.638992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:26.638992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:26.638992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:26.638992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:26.638992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:26.638992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:26.638992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:26.638992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:26.638992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:26.638992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:26.638992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:26.638992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:26.638992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:26.639897 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:26.639897 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:26.639897 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:26.639897 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:26.646341 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 408/823] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:26.647805 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91036b62-2087-4cff-99d8-94e83b5ed4a1 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:26.650820 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46bc259-3076-4e0f-adbc-fdc716c09282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:26.651701 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46bc259-3076-4e0f-adbc-fdc716c09282 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:26.651932 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46bc259-3076-4e0f-adbc-fdc716c09282 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:26.652203 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46bc259-3076-4e0f-adbc-fdc716c09282 None admin] PATH_INFO: `/v3/services/network` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:26.653208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46bc259-3076-4e0f-adbc-fdc716c09282 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:26.654235 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46bc259-3076-4e0f-adbc-fdc716c09282 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:26.667154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-c46bc259-3076-4e0f-adbc-fdc716c09282 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 23:02:26.667154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:26.667154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 23:02:26.667154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 23:02:26.667154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 23:02:26.667154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 23:02:26.667154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 23:02:26.667154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:26.667154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 23:02:26.667154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:26.667154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 23:02:26.667154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:26.667154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:26.667154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:26.667154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:26.667154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:26.667154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:26.668208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:26.668208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:26.668208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:26.668208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:26.668208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:26.668208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:26.668208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 23:02:26.668208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 23:02:26.668208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 23:02:26.668208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 23:02:26.668208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:26.668208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:26.668208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:26.668208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:26.668208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:26.668208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:26.669038 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:26.669038 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:26.669038 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:26.669038 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:26.669038 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:26.669038 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:26.669038 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:26.669038 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:26.669038 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:26.669038 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:26.669038 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 23:02:26.669038 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:26.669038 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 23:02:26.669038 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:26.669038 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 416/824] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:02:26 2022] GET /identity/v3/services/network => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:26.669809 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46bc259-3076-4e0f-adbc-fdc716c09282 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:26.705751 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:26.765691 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91036b62-2087-4cff-99d8-94e83b5ed4a1 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:26.776687 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:26.798160 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46bc259-3076-4e0f-adbc-fdc716c09282 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:26.805836 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec4702d-6252-48ed-8a46-846d57bcd693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:26.806842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4702d-6252-48ed-8a46-846d57bcd693 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:26.806842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4702d-6252-48ed-8a46-846d57bcd693 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:26.807044 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4702d-6252-48ed-8a46-846d57bcd693 None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:26.807774 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec4702d-6252-48ed-8a46-846d57bcd693 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:26.808262 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec4702d-6252-48ed-8a46-846d57bcd693 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:26.809738 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec4702d-6252-48ed-8a46-846d57bcd693 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:26.826183 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 409/825] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:26.826183 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec4702d-6252-48ed-8a46-846d57bcd693 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:26.843958 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b638a-b865-4ea6-9413-48c412ca6681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:26.844634 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b638a-b865-4ea6-9413-48c412ca6681 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:26.844905 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b638a-b865-4ea6-9413-48c412ca6681 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:26.845173 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5b638a-b865-4ea6-9413-48c412ca6681 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:26.845929 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5b638a-b865-4ea6-9413-48c412ca6681 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:26.846238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5b638a-b865-4ea6-9413-48c412ca6681 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:26.847655 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5b638a-b865-4ea6-9413-48c412ca6681 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:26.858455 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 417/826] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:02:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:26.859299 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b638a-b865-4ea6-9413-48c412ca6681 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:26.950004 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec4702d-6252-48ed-8a46-846d57bcd693 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:26.960687 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:26.971368 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5b638a-b865-4ea6-9413-48c412ca6681 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:26.991489 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ad4fc-e59e-4844-8276-1785207ca58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:26.992210 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ad4fc-e59e-4844-8276-1785207ca58a None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:26.992546 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ad4fc-e59e-4844-8276-1785207ca58a None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:26.992829 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ad4fc-e59e-4844-8276-1785207ca58a None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:26.993577 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ad4fc-e59e-4844-8276-1785207ca58a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:26.993942 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ad4fc-e59e-4844-8276-1785207ca58a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:26.995365 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ad4fc-e59e-4844-8276-1785207ca58a None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:27.011941 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f07eda-ac46-4159-8ca6-e803d6e3b5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:27.012658 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f07eda-ac46-4159-8ca6-e803d6e3b5da None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:27.012942 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f07eda-ac46-4159-8ca6-e803d6e3b5da None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:27.013216 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f07eda-ac46-4159-8ca6-e803d6e3b5da None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:27.013986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f07eda-ac46-4159-8ca6-e803d6e3b5da None admin] RBAC: Adding query filter params (name=network) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:27.014244 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 410/827] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:27.014340 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f07eda-ac46-4159-8ca6-e803d6e3b5da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:27.015218 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ad4fc-e59e-4844-8276-1785207ca58a None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:27.015672 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f07eda-ac46-4159-8ca6-e803d6e3b5da None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:27.022857 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 418/828] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Jul 13 23:02:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:27.023778 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f07eda-ac46-4159-8ca6-e803d6e3b5da None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:27.049666 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:27.175572 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ad4fc-e59e-4844-8276-1785207ca58a None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:27.181459 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f07eda-ac46-4159-8ca6-e803d6e3b5da None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:27.214507 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf95980-52a8-4063-b834-6828263e0717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:27.215270 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf95980-52a8-4063-b834-6828263e0717 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:27.215577 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf95980-52a8-4063-b834-6828263e0717 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:27.215860 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf95980-52a8-4063-b834-6828263e0717 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:27.220935 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d728ea6b-5214-462f-aff5-5817c8035a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:27.222053 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d728ea6b-5214-462f-aff5-5817c8035a87 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:27.222552 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d728ea6b-5214-462f-aff5-5817c8035a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:27.222892 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf95980-52a8-4063-b834-6828263e0717 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:27.223209 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d728ea6b-5214-462f-aff5-5817c8035a87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:27.224403 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf95980-52a8-4063-b834-6828263e0717 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:27.238181 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d728ea6b-5214-462f-aff5-5817c8035a87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:27.238622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-bbf95980-52a8-4063-b834-6828263e0717 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:27.238622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:27.238622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:27.238622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:27.238622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:27.238622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:27.238622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:27.238622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:27.238622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:27.238622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:27.238622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:27.238622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:27.238622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 23:02:27.238622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 23:02:27.238622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 23:02:27.239460 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 23:02:27.239460 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:27.239460 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:27.239460 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:27.239460 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:27.239460 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:27.239460 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:27.239460 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:27.239460 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:27.239460 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:27.239460 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:27.239460 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:27.239460 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:27.239460 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:27.239460 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:27.239460 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:27.239460 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:27.240404 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 23:02:27.240404 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 23:02:27.240404 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 23:02:27.240404 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 23:02:27.240404 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 23:02:27.240404 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 23:02:27.240404 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:27.240404 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:27.240404 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 411/829] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:02:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:27.241307 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf95980-52a8-4063-b834-6828263e0717 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:27.241727 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d728ea6b-5214-462f-aff5-5817c8035a87 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:27.265090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-d728ea6b-5214-462f-aff5-5817c8035a87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:27.265090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:27.265090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:27.265090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:27.265090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:27.265090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:27.265090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:27.265090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:27.265090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:27.265090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:27.265090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:27.265090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:27.265090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:27.265090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:27.265090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:27.266224 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:27.266224 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:27.266224 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:27.266224 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:27.266224 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:27.266224 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:27.266224 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:27.266224 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:27.266224 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:27.266224 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:27.266224 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:27.266224 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:27.266224 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:27.266224 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:27.266224 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:27.266224 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:27.266224 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:27.267094 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:27.267094 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:27.267094 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:27.267094 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:27.267094 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 419/830] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:27.268567 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d728ea6b-5214-462f-aff5-5817c8035a87 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:27.287345 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:27.377610 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:27.395228 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf95980-52a8-4063-b834-6828263e0717 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:27.395621 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d728ea6b-5214-462f-aff5-5817c8035a87 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:27.438667 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7698b09-5899-4878-adde-dbbf217dc53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:27.439323 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7698b09-5899-4878-adde-dbbf217dc53f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:27.439628 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7698b09-5899-4878-adde-dbbf217dc53f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:27.439900 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7698b09-5899-4878-adde-dbbf217dc53f None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:27.440604 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7698b09-5899-4878-adde-dbbf217dc53f None admin] RBAC: Adding query filter params (type=network) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:27.440748 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf9d2c3-ed1e-4495-826b-ccfb64bf012d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:27.440930 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7698b09-5899-4878-adde-dbbf217dc53f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:27.441609 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9d2c3-ed1e-4495-826b-ccfb64bf012d None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:27.441890 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9d2c3-ed1e-4495-826b-ccfb64bf012d None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:27.442161 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9d2c3-ed1e-4495-826b-ccfb64bf012d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:27.442260 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7698b09-5899-4878-adde-dbbf217dc53f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:27.449110 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 420/831] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Jul 13 23:02:27 2022] GET /identity/v3/services?type=network => generated 350 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:27.449748 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7698b09-5899-4878-adde-dbbf217dc53f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:27.456807 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf9d2c3-ed1e-4495-826b-ccfb64bf012d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:27.458299 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf9d2c3-ed1e-4495-826b-ccfb64bf012d None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:27.473548 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-1bf9d2c3-ed1e-4495-826b-ccfb64bf012d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:27.473548 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:27.473548 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:27.473548 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:27.473548 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:27.473548 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:27.473548 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:27.473548 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:27.473548 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:27.473548 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:27.473548 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:27.473548 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:27.473548 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 23:02:27.473548 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 23:02:27.473548 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 23:02:27.475003 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 23:02:27.475003 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:27.475003 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:27.475003 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:27.475003 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:27.475003 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:27.475003 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:27.475003 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:27.475003 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:27.475003 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:27.475003 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:27.475003 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:27.475003 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:27.475003 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:27.475003 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:27.475003 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:27.475003 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:27.475941 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 23:02:27.475941 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 23:02:27.475941 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 23:02:27.475941 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 23:02:27.475941 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 23:02:27.475941 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 23:02:27.475941 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:27.475941 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:27.475941 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 412/832] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:02:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:27.475941 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf9d2c3-ed1e-4495-826b-ccfb64bf012d None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:27.531659 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:27.565216 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7698b09-5899-4878-adde-dbbf217dc53f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:27.604216 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf9d2c3-ed1e-4495-826b-ccfb64bf012d None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:27.604841 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d832f-ac4a-4cbb-ae2a-246a6b07a808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:27.605509 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d832f-ac4a-4cbb-ae2a-246a6b07a808 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:27.605800 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d832f-ac4a-4cbb-ae2a-246a6b07a808 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:27.606067 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d832f-ac4a-4cbb-ae2a-246a6b07a808 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:27.606862 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8d832f-ac4a-4cbb-ae2a-246a6b07a808 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:27.607213 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8d832f-ac4a-4cbb-ae2a-246a6b07a808 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:27.608671 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8d832f-ac4a-4cbb-ae2a-246a6b07a808 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:27.622238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 421/833] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jul 13 23:02:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:27.627726 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d832f-ac4a-4cbb-ae2a-246a6b07a808 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:27.649363 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9ab88-0b21-4fc3-a69c-2a754bff2ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:27.650140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9ab88-0b21-4fc3-a69c-2a754bff2ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:27.650672 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9ab88-0b21-4fc3-a69c-2a754bff2ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:27.650958 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b9ab88-0b21-4fc3-a69c-2a754bff2ca1 None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:27.651728 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b9ab88-0b21-4fc3-a69c-2a754bff2ca1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:27.652036 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b9ab88-0b21-4fc3-a69c-2a754bff2ca1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:27.653431 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b9ab88-0b21-4fc3-a69c-2a754bff2ca1 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:27.671721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 413/834] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:27.672581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9ab88-0b21-4fc3-a69c-2a754bff2ca1 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:27.685612 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:27.737917 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d832f-ac4a-4cbb-ae2a-246a6b07a808 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:27.772986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:27.779245 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1ded3a-20da-4369-aca4-c01e160e824a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:27.779936 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1ded3a-20da-4369-aca4-c01e160e824a None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:27.780238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1ded3a-20da-4369-aca4-c01e160e824a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:27.780506 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1ded3a-20da-4369-aca4-c01e160e824a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:27.781239 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1ded3a-20da-4369-aca4-c01e160e824a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:27.782645 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1ded3a-20da-4369-aca4-c01e160e824a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:27.792975 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b9ab88-0b21-4fc3-a69c-2a754bff2ca1 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:27.807959 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 422/835] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Jul 13 23:02:27 2022] POST /identity/v3/endpoints => generated 327 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:02:27.809056 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1ded3a-20da-4369-aca4-c01e160e824a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:27.845943 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69921b6a-ef2c-473c-8248-5b31c38940b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:27.846627 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69921b6a-ef2c-473c-8248-5b31c38940b5 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:27.846911 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69921b6a-ef2c-473c-8248-5b31c38940b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:27.847332 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69921b6a-ef2c-473c-8248-5b31c38940b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:27.848032 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69921b6a-ef2c-473c-8248-5b31c38940b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:27.848342 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69921b6a-ef2c-473c-8248-5b31c38940b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:27.849830 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69921b6a-ef2c-473c-8248-5b31c38940b5 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:27.869912 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 414/836] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jul 13 23:02:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:27.869912 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60100a9f-e607-4523-acb1-d5af8aa1116f None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:27.869912 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60100a9f-e607-4523-acb1-d5af8aa1116f None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:27.869912 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60100a9f-e607-4523-acb1-d5af8aa1116f None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:27.871701 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 415/837] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:27.872846 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60100a9f-e607-4523-acb1-d5af8aa1116f None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:27.962238 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:27.970395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1ded3a-20da-4369-aca4-c01e160e824a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:28.019050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60100a9f-e607-4523-acb1-d5af8aa1116f None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:28.019388 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36496e6-926a-428a-a775-0aaf20a479a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:28.022176 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36496e6-926a-428a-a775-0aaf20a479a2 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:28.022176 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36496e6-926a-428a-a775-0aaf20a479a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:28.022176 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36496e6-926a-428a-a775-0aaf20a479a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:28.022176 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36496e6-926a-428a-a775-0aaf20a479a2 None admin] RBAC: Adding query filter params (role.id=8cfd0b6e71e14e0983c3eb4d7478e027, scope.project.id=ab0a5581e4e444c293815e1304e20c05, user.id=4789384d158b4995bd30e1e37e30f5f1) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:28.022176 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36496e6-926a-428a-a775-0aaf20a479a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:28.023326 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36496e6-926a-428a-a775-0aaf20a479a2 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:28.031985 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 423/838] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:02:27 2022] GET /identity/v3/role_assignments?user.id=4789384d158b4995bd30e1e37e30f5f1&scope.project.id=ab0a5581e4e444c293815e1304e20c05&role.id=8cfd0b6e71e14e0983c3eb4d7478e027 => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:28.033176 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36496e6-926a-428a-a775-0aaf20a479a2 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:28.076187 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c740fec3-c67e-4477-a05f-357eef50cdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:28.077032 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740fec3-c67e-4477-a05f-357eef50cdc5 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:28.077302 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740fec3-c67e-4477-a05f-357eef50cdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:28.077581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740fec3-c67e-4477-a05f-357eef50cdc5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:28.087323 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c740fec3-c67e-4477-a05f-357eef50cdc5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:28.088876 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c740fec3-c67e-4477-a05f-357eef50cdc5 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:28.115382 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-c740fec3-c67e-4477-a05f-357eef50cdc5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 13 23:02:28.115382 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:28.115382 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:28.115382 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:28.115382 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:28.115382 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:28.115382 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:28.115382 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:28.115382 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:28.115382 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:28.115382 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:28.115382 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:28.115382 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 23:02:28.115382 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 23:02:28.115382 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 23:02:28.115382 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 23:02:28.117054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:28.117054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:28.117054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 23:02:28.117054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:28.117054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 23:02:28.117054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:28.117054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:28.117054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:28.117054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:28.117054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:28.117054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:28.117054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:28.117054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:28.117054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:28.117054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:28.117054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:28.117054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:28.117054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:28.117989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:28.117989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:28.117989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 23:02:28.117989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 23:02:28.117989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 23:02:28.117989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 23:02:28.117989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 23:02:28.117989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 23:02:28.117989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 13 23:02:28.117989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:28.117989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 416/839] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jul 13 23:02:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:28.118987 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:28.119091 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6243b37-2386-4a92-ae13-2e029f052201 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:28.119173 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6243b37-2386-4a92-ae13-2e029f052201 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:28.119770 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6243b37-2386-4a92-ae13-2e029f052201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:28.146055 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-d6243b37-2386-4a92-ae13-2e029f052201 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:28.147982 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-d6243b37-2386-4a92-ae13-2e029f052201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:28.180954 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36496e6-926a-428a-a775-0aaf20a479a2 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:28.224767 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7712b8c-a08d-4bea-962f-50df0e4a3a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:28.226224 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7712b8c-a08d-4bea-962f-50df0e4a3a5b None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:28.226511 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7712b8c-a08d-4bea-962f-50df0e4a3a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:28.227500 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7712b8c-a08d-4bea-962f-50df0e4a3a5b None admin] PATH_INFO: `/v3/projects/ab0a5581e4e444c293815e1304e20c05/users/a32702a67a7b49ffa153fc59251fd169/roles/92417b14d4984ed9956dcd14f97564f3` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:28.250932 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 417/840] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:02:28.251208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7712b8c-a08d-4bea-962f-50df0e4a3a5b None admin] RBAC: Authorizing `identity:create_grant(project_id=ab0a5581e4e444c293815e1304e20c05, user_id=a32702a67a7b49ffa153fc59251fd169, role_id=92417b14d4984ed9956dcd14f97564f3)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:28.251365 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6243b37-2386-4a92-ae13-2e029f052201 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:28.251519 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7712b8c-a08d-4bea-962f-50df0e4a3a5b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:28.267710 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 424/841] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jul 13 23:02:28 2022] PUT /identity/v3/projects/ab0a5581e4e444c293815e1304e20c05/users/a32702a67a7b49ffa153fc59251fd169/roles/92417b14d4984ed9956dcd14f97564f3 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:02:28.290882 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7712b8c-a08d-4bea-962f-50df0e4a3a5b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:28.376196 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:28.465446 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6243b37-2386-4a92-ae13-2e029f052201 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:28.508862 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeae634-c1e1-46e9-9810-5f243d9a3954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:28.509675 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7712b8c-a08d-4bea-962f-50df0e4a3a5b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:28.509911 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeae634-c1e1-46e9-9810-5f243d9a3954 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:28.510353 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeae634-c1e1-46e9-9810-5f243d9a3954 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:28.510790 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeae634-c1e1-46e9-9810-5f243d9a3954 None admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:28.512664 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:28.513476 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeae634-c1e1-46e9-9810-5f243d9a3954 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:28.514012 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeae634-c1e1-46e9-9810-5f243d9a3954 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:28.517943 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeae634-c1e1-46e9-9810-5f243d9a3954 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:28.560132 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 418/842] 10.222.0.60 () {56 vars in 1135 bytes} [Wed Jul 13 23:02:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:28.568469 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeae634-c1e1-46e9-9810-5f243d9a3954 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:28.575203 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2a11c-3ab6-4bbd-a7d9-7cdd060820e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:28.576576 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2a11c-3ab6-4bbd-a7d9-7cdd060820e0 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:28.577049 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2a11c-3ab6-4bbd-a7d9-7cdd060820e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:28.577502 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2a11c-3ab6-4bbd-a7d9-7cdd060820e0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:28.578648 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb2a11c-3ab6-4bbd-a7d9-7cdd060820e0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:28.581339 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb2a11c-3ab6-4bbd-a7d9-7cdd060820e0 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:28.606776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-bbb2a11c-3ab6-4bbd-a7d9-7cdd060820e0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 23:02:28.606776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:28.606776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 23:02:28.606776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 23:02:28.606776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 23:02:28.606776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 23:02:28.606776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 23:02:28.606776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:28.606776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 23:02:28.606776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:28.606776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 23:02:28.606776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:28.606776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:28.606776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:28.606776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:28.606776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:28.606776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:28.607863 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:28.607863 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:28.607863 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:28.607863 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:28.607863 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:28.607863 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:28.607863 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 23:02:28.607863 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 23:02:28.607863 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 23:02:28.607863 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 23:02:28.607863 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:28.607863 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:28.607863 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:28.607863 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:28.607863 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:28.607863 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:28.608712 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:28.608712 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:28.608712 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:28.608712 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:28.608712 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:28.608712 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:28.608712 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:28.608712 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:28.608712 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:28.608712 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:28.608712 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 23:02:28.608712 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:28.608712 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 23:02:28.608712 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:28.609441 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 425/843] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jul 13 23:02:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:02:28.623553 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2a11c-3ab6-4bbd-a7d9-7cdd060820e0 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:28.726296 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeae634-c1e1-46e9-9810-5f243d9a3954 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:28.750803 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:28.752087 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2a11c-3ab6-4bbd-a7d9-7cdd060820e0 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:28.791070 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab723988-b5df-4b23-91e9-be685a9472b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:28.792572 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab723988-b5df-4b23-91e9-be685a9472b0 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:28.793396 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab723988-b5df-4b23-91e9-be685a9472b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:28.794302 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab723988-b5df-4b23-91e9-be685a9472b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:28.796265 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d087ceb3-843d-4d28-8e1c-72f8fec1e287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:28.797026 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087ceb3-843d-4d28-8e1c-72f8fec1e287 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:28.797371 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087ceb3-843d-4d28-8e1c-72f8fec1e287 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:28.797709 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087ceb3-843d-4d28-8e1c-72f8fec1e287 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:28.798500 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d087ceb3-843d-4d28-8e1c-72f8fec1e287 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:28.798915 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d087ceb3-843d-4d28-8e1c-72f8fec1e287 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:28.800463 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d087ceb3-843d-4d28-8e1c-72f8fec1e287 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:28.807570 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 426/844] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jul 13 23:02:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:28.815919 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d087ceb3-843d-4d28-8e1c-72f8fec1e287 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:28.854708 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:28.924833 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab723988-b5df-4b23-91e9-be685a9472b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:28.927755 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab723988-b5df-4b23-91e9-be685a9472b0 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:28.941825 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d087ceb3-843d-4d28-8e1c-72f8fec1e287 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:29.012764 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91ce19-0533-4f90-8967-ef0d1f9e9b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:29.013898 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91ce19-0533-4f90-8967-ef0d1f9e9b22 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:29.014390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91ce19-0533-4f90-8967-ef0d1f9e9b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:29.014877 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91ce19-0533-4f90-8967-ef0d1f9e9b22 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:29.016253 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b91ce19-0533-4f90-8967-ef0d1f9e9b22 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:29.016939 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b91ce19-0533-4f90-8967-ef0d1f9e9b22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:29.019689 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b91ce19-0533-4f90-8967-ef0d1f9e9b22 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:29.036136 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 427/845] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jul 13 23:02:28 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:02:29.101489 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 419/846] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:02:28 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 23:02:29.107810 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91ce19-0533-4f90-8967-ef0d1f9e9b22 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:29.261571 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:29.275286 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91ce19-0533-4f90-8967-ef0d1f9e9b22 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:29.335088 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c781e-5b93-4072-af7b-afefbeb1b29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:29.336483 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c781e-5b93-4072-af7b-afefbeb1b29d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:29.336972 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c781e-5b93-4072-af7b-afefbeb1b29d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:29.337449 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c781e-5b93-4072-af7b-afefbeb1b29d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:29.362547 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c781e-5b93-4072-af7b-afefbeb1b29d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:29.364084 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c781e-5b93-4072-af7b-afefbeb1b29d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:29.383306 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-eb4c781e-5b93-4072-af7b-afefbeb1b29d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:29.383306 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:29.383306 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:29.383306 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:29.383306 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:29.383306 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:29.383306 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:29.383306 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:29.383306 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:29.383306 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:29.383306 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:29.383306 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:29.383306 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:29.383306 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:29.383306 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:29.384390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:29.384390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:29.384390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:29.384390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:29.384390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:29.384390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:29.384390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:29.384390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:29.384390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:29.384390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:29.384390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:29.384390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:29.384390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:29.384390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:29.384390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:29.384390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:29.384390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:29.385402 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:29.385402 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:29.385402 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:29.385402 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:29.385402 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 428/847] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:29.391740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab723988-b5df-4b23-91e9-be685a9472b0 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:29.415215 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:29.568236 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab723988-b5df-4b23-91e9-be685a9472b0 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:29.619103 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1007989f-0f7d-40bc-ab16-d2da4f646356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:29.619720 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007989f-0f7d-40bc-ab16-d2da4f646356 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:29.620012 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007989f-0f7d-40bc-ab16-d2da4f646356 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:29.620289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007989f-0f7d-40bc-ab16-d2da4f646356 None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:29.621027 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1007989f-0f7d-40bc-ab16-d2da4f646356 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:29.621334 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1007989f-0f7d-40bc-ab16-d2da4f646356 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:29.622723 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1007989f-0f7d-40bc-ab16-d2da4f646356 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:29.635715 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 420/848] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:29.641388 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c781e-5b93-4072-af7b-afefbeb1b29d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:29.751886 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c781e-5b93-4072-af7b-afefbeb1b29d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:29.790751 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c569dd-b094-4348-879d-2fc46e27fa8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:29.792000 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c569dd-b094-4348-879d-2fc46e27fa8b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:29.792497 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c569dd-b094-4348-879d-2fc46e27fa8b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:29.792976 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c569dd-b094-4348-879d-2fc46e27fa8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:29.800649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c569dd-b094-4348-879d-2fc46e27fa8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:29.802422 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c569dd-b094-4348-879d-2fc46e27fa8b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:29.819338 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-16c569dd-b094-4348-879d-2fc46e27fa8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:29.819338 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:29.819338 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:29.819338 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:29.819338 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:29.819338 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:29.819338 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:29.819338 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:29.819338 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:29.819338 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:29.819338 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:29.819338 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:29.819338 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 23:02:29.819338 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 23:02:29.819338 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 23:02:29.820395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 23:02:29.820395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:29.820395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:29.820395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:29.820395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:29.820395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:29.820395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:29.820395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:29.820395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:29.820395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:29.820395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:29.820395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:29.820395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:29.820395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:29.820395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:29.820395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:29.820395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:29.821549 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 23:02:29.821549 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 23:02:29.821549 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 23:02:29.821549 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 23:02:29.821549 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 23:02:29.821549 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 23:02:29.821549 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:29.821549 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:29.824238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 429/849] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:02:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:29.830832 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1007989f-0f7d-40bc-ab16-d2da4f646356 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:29.906425 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:29.940940 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1007989f-0f7d-40bc-ab16-d2da4f646356 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:29.977769 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1588db0-28e0-489f-b564-8194b536b2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:29.978447 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1588db0-28e0-489f-b564-8194b536b2a4 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:29.978733 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1588db0-28e0-489f-b564-8194b536b2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:29.979011 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1588db0-28e0-489f-b564-8194b536b2a4 None admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:29.979853 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1588db0-28e0-489f-b564-8194b536b2a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:29.980227 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1588db0-28e0-489f-b564-8194b536b2a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:29.981778 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1588db0-28e0-489f-b564-8194b536b2a4 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:29.997398 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 421/850] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jul 13 23:02:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:02:30.003507 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c569dd-b094-4348-879d-2fc46e27fa8b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:30.013645 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:30.113615 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c569dd-b094-4348-879d-2fc46e27fa8b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:30.150224 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0792c489-0bb4-4f40-bbcb-f0c2aa70fd07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:30.151253 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792c489-0bb4-4f40-bbcb-f0c2aa70fd07 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:30.151782 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792c489-0bb4-4f40-bbcb-f0c2aa70fd07 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:30.152274 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792c489-0bb4-4f40-bbcb-f0c2aa70fd07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:30.153305 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0792c489-0bb4-4f40-bbcb-f0c2aa70fd07 None admin] RBAC: Adding query filter params (role.id=8cfd0b6e71e14e0983c3eb4d7478e027, scope.project.id=ab0a5581e4e444c293815e1304e20c05, user.id=786fc44611804d3ca6280ffcab415569) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:30.153917 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0792c489-0bb4-4f40-bbcb-f0c2aa70fd07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:30.155572 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0792c489-0bb4-4f40-bbcb-f0c2aa70fd07 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:30.163066 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 430/851] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:02:30 2022] GET /identity/v3/role_assignments?user.id=786fc44611804d3ca6280ffcab415569&scope.project.id=ab0a5581e4e444c293815e1304e20c05&role.id=8cfd0b6e71e14e0983c3eb4d7478e027 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:30.779341 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28be888-a9b2-44ee-8e67-0b616d82121c None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:30.779940 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28be888-a9b2-44ee-8e67-0b616d82121c None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:30.779940 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28be888-a9b2-44ee-8e67-0b616d82121c None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:30.781648 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 422/852] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:30.787896 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d173d2-ec6d-4cc3-a04d-3ea7744d4d0c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:30.788394 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d173d2-ec6d-4cc3-a04d-3ea7744d4d0c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:30.788492 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d173d2-ec6d-4cc3-a04d-3ea7744d4d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:30.806650 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-a8d173d2-ec6d-4cc3-a04d-3ea7744d4d0c None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:30.807911 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a8d173d2-ec6d-4cc3-a04d-3ea7744d4d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:30.884238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:30.890360 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 431/853] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:30 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 23:02:30.914471 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28be888-a9b2-44ee-8e67-0b616d82121c None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:31.030413 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28be888-a9b2-44ee-8e67-0b616d82121c None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:31.064219 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:31.069531 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81204817-c8c7-4e60-af8f-38201f4804b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:31.070414 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81204817-c8c7-4e60-af8f-38201f4804b8 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:31.071514 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81204817-c8c7-4e60-af8f-38201f4804b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:31.071916 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81204817-c8c7-4e60-af8f-38201f4804b8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:31.084241 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81204817-c8c7-4e60-af8f-38201f4804b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:02:31.092405 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-81204817-c8c7-4e60-af8f-38201f4804b8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:02:31.092405 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:31.092405 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:31.092405 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:31.092405 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:31.092405 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:31.092405 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:31.092405 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:31.092405 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:31.092405 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:31.092405 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:31.092405 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:31.092405 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:02:31.092405 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:02:31.092405 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:02:31.092405 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:02:31.093394 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:02:31.093394 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:02:31.093394 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:02:31.093394 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:02:31.093394 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:31.093394 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:31.093394 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:31.093394 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:31.093394 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:31.093394 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:31.093394 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:31.093394 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:31.093394 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:31.093394 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:31.093394 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:31.093394 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:31.094222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:31.094222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:31.094222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:31.094222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:31.094222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:02:31.094222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:02:31.094222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:02:31.094222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:02:31.094222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:02:31.094222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:02:31.094222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:02:31.094222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:31.094222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 423/854] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jul 13 23:02:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:31.101335 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d173d2-ec6d-4cc3-a04d-3ea7744d4d0c None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:31.280871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d173d2-ec6d-4cc3-a04d-3ea7744d4d0c None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:31.318959 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3cb504-4ce8-4f07-b42e-e2c8938964a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:31.320070 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3cb504-4ce8-4f07-b42e-e2c8938964a1 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:31.320577 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3cb504-4ce8-4f07-b42e-e2c8938964a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:31.321319 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3cb504-4ce8-4f07-b42e-e2c8938964a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:31.322298 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3cb504-4ce8-4f07-b42e-e2c8938964a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:31.322847 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3cb504-4ce8-4f07-b42e-e2c8938964a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:31.324478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3cb504-4ce8-4f07-b42e-e2c8938964a1 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:31.334829 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 432/855] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jul 13 23:02:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:02:31.340344 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81204817-c8c7-4e60-af8f-38201f4804b8 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:31.412956 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16027d3a-6724-4f45-b786-57041c9ed90e None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:31.413724 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16027d3a-6724-4f45-b786-57041c9ed90e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:31.414267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16027d3a-6724-4f45-b786-57041c9ed90e None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:31.416242 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 433/856] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:02:31.424468 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5828a7fd-8173-4cdb-bb6c-24f79f88c043 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:31.424788 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5828a7fd-8173-4cdb-bb6c-24f79f88c043 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:31.425039 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5828a7fd-8173-4cdb-bb6c-24f79f88c043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:31.448492 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-5828a7fd-8173-4cdb-bb6c-24f79f88c043 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:31.450371 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5828a7fd-8173-4cdb-bb6c-24f79f88c043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:31.507054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81204817-c8c7-4e60-af8f-38201f4804b8 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:31.556684 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:31.576677 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad9b58e-21d2-4546-b77c-f499bd19cae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:31.578012 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad9b58e-21d2-4546-b77c-f499bd19cae0 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:31.578512 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad9b58e-21d2-4546-b77c-f499bd19cae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:31.578991 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad9b58e-21d2-4546-b77c-f499bd19cae0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:31.580782 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 434/857] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:02:31.599111 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad9b58e-21d2-4546-b77c-f499bd19cae0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:31.601850 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad9b58e-21d2-4546-b77c-f499bd19cae0 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:31.603226 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5828a7fd-8173-4cdb-bb6c-24f79f88c043 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:31.636730 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-fad9b58e-21d2-4546-b77c-f499bd19cae0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:31.636730 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:31.636730 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:31.636730 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:31.636730 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:31.636730 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:31.636730 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:31.636730 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:31.636730 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:31.636730 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:31.636730 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:31.636730 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:31.636730 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:31.636730 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:31.636730 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:31.637673 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:31.637673 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:31.637673 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:31.637673 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:31.637673 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:31.637673 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:31.637673 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:31.637673 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:31.637673 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:31.637673 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:31.637673 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:31.637673 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:31.637673 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:31.637673 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:31.637673 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:31.637673 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:31.637673 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:31.638547 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:31.638547 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:31.638547 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:31.638547 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:31.639353 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 424/858] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:31.644633 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad9b58e-21d2-4546-b77c-f499bd19cae0 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:31.708286 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:31.718793 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5828a7fd-8173-4cdb-bb6c-24f79f88c043 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:31.757731 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7349d73b-c6d0-4247-8330-20aeec85f386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:31.759017 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349d73b-c6d0-4247-8330-20aeec85f386 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:31.759520 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349d73b-c6d0-4247-8330-20aeec85f386 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:31.760059 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349d73b-c6d0-4247-8330-20aeec85f386 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:31.760942 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7349d73b-c6d0-4247-8330-20aeec85f386 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:31.762653 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7349d73b-c6d0-4247-8330-20aeec85f386 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:31.778117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad9b58e-21d2-4546-b77c-f499bd19cae0 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:31.785803 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-7349d73b-c6d0-4247-8330-20aeec85f386 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 23:02:31.785803 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:31.785803 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 23:02:31.785803 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 23:02:31.785803 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 23:02:31.785803 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 23:02:31.785803 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 23:02:31.785803 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:31.785803 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 23:02:31.785803 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:31.785803 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 23:02:31.785803 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:31.785803 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:31.785803 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:31.785803 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:31.785803 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:31.786818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:31.786818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:31.786818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:31.786818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:31.786818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:31.786818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:31.786818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:31.786818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 23:02:31.786818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 23:02:31.786818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 23:02:31.786818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 23:02:31.786818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:31.786818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:31.786818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:31.786818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:31.786818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:31.787818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:31.787818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:31.787818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:31.787818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:31.787818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:31.787818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:31.787818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:31.787818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:31.787818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:31.787818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:31.787818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:31.787818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 23:02:31.787818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:31.787818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 23:02:31.787818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:31.791659 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 435/859] 10.222.0.60 () {56 vars in 1132 bytes} [Wed Jul 13 23:02:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:02:31.798508 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7349d73b-c6d0-4247-8330-20aeec85f386 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:31.840824 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a55aa-9ecf-493b-b528-feb3cbc2ef25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:31.841905 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a55aa-9ecf-493b-b528-feb3cbc2ef25 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:31.842361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a55aa-9ecf-493b-b528-feb3cbc2ef25 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:31.842924 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a55aa-9ecf-493b-b528-feb3cbc2ef25 None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:31.845463 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a55aa-9ecf-493b-b528-feb3cbc2ef25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:31.846035 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a55aa-9ecf-493b-b528-feb3cbc2ef25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:31.851003 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a55aa-9ecf-493b-b528-feb3cbc2ef25 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:31.892223 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:31.892544 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 425/860] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:31.901051 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a55aa-9ecf-493b-b528-feb3cbc2ef25 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:31.943485 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7349d73b-c6d0-4247-8330-20aeec85f386 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:32.015566 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b49b937-a31d-494e-84eb-2e33efac8301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:32.015566 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b49b937-a31d-494e-84eb-2e33efac8301 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:32.015566 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b49b937-a31d-494e-84eb-2e33efac8301 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:32.015566 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b49b937-a31d-494e-84eb-2e33efac8301 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:32.016007 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b49b937-a31d-494e-84eb-2e33efac8301 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:32.016007 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b49b937-a31d-494e-84eb-2e33efac8301 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:32.016007 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b49b937-a31d-494e-84eb-2e33efac8301 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:32.027991 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 436/861] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Jul 13 23:02:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:02:32.033835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b49b937-a31d-494e-84eb-2e33efac8301 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:32.080066 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a55aa-9ecf-493b-b528-feb3cbc2ef25 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:32.082732 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:32.123470 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80398f1-636f-41e9-9eeb-8359a56e1cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:32.125470 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80398f1-636f-41e9-9eeb-8359a56e1cbf None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:32.125754 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80398f1-636f-41e9-9eeb-8359a56e1cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:32.126030 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80398f1-636f-41e9-9eeb-8359a56e1cbf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:32.145641 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80398f1-636f-41e9-9eeb-8359a56e1cbf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:32.147479 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80398f1-636f-41e9-9eeb-8359a56e1cbf None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:32.160403 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b49b937-a31d-494e-84eb-2e33efac8301 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:32.165700 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-c80398f1-636f-41e9-9eeb-8359a56e1cbf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 23:02:32.165700 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:32.165700 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:32.165700 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:32.165700 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:32.165700 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:32.165700 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:32.165700 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:32.165700 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:32.165700 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:32.165700 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:32.165700 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:32.165700 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 23:02:32.165700 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 23:02:32.165700 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 23:02:32.165700 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 23:02:32.167747 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:32.167747 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:32.167747 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 23:02:32.167747 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:32.167747 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 23:02:32.167747 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:32.167747 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:32.167747 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:32.167747 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:32.167747 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:32.167747 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:32.167747 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:32.167747 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:32.167747 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:32.167747 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:32.167747 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:32.167747 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:32.167747 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:32.170793 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:32.170793 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:32.170793 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 23:02:32.170793 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 23:02:32.170793 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 23:02:32.170793 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 23:02:32.170793 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 23:02:32.170793 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 23:02:32.170793 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 23:02:32.170793 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:32.172438 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 426/862] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Jul 13 23:02:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:32.173432 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab7c3d-4eb1-4877-9ef2-fd791b85dd19 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:32.173722 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab7c3d-4eb1-4877-9ef2-fd791b85dd19 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:32.173993 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab7c3d-4eb1-4877-9ef2-fd791b85dd19 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:32.182591 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 427/863] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:32.182591 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ab7c3d-4eb1-4877-9ef2-fd791b85dd19 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:32.198593 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:32.218789 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c44253a-d41d-4c12-b6e5-efa1fa7dca76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:32.219452 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44253a-d41d-4c12-b6e5-efa1fa7dca76 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:32.219942 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44253a-d41d-4c12-b6e5-efa1fa7dca76 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:32.220220 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44253a-d41d-4c12-b6e5-efa1fa7dca76 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:32.221485 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c44253a-d41d-4c12-b6e5-efa1fa7dca76 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:32.221931 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c44253a-d41d-4c12-b6e5-efa1fa7dca76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:32.223332 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c44253a-d41d-4c12-b6e5-efa1fa7dca76 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:32.240056 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 437/864] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Jul 13 23:02:32 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:02:32.243575 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644ed6d-a64c-486d-a2fb-ec9d2fbee02e None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:32.243575 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644ed6d-a64c-486d-a2fb-ec9d2fbee02e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:32.243575 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644ed6d-a64c-486d-a2fb-ec9d2fbee02e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:32.266799 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-3644ed6d-a64c-486d-a2fb-ec9d2fbee02e None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:32.271957 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3644ed6d-a64c-486d-a2fb-ec9d2fbee02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:32.417979 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ab7c3d-4eb1-4877-9ef2-fd791b85dd19 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:32.449855 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 438/865] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:02:32.463228 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dffac7d-9eb4-4c28-b3d2-d82d371b1808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:32.463951 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffac7d-9eb4-4c28-b3d2-d82d371b1808 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:32.464227 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffac7d-9eb4-4c28-b3d2-d82d371b1808 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:32.464493 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffac7d-9eb4-4c28-b3d2-d82d371b1808 None admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:32.465344 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dffac7d-9eb4-4c28-b3d2-d82d371b1808 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:32.465650 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dffac7d-9eb4-4c28-b3d2-d82d371b1808 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:32.467116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dffac7d-9eb4-4c28-b3d2-d82d371b1808 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:32.472964 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e256f-8b64-4342-9c7d-b1bf3b8e68c7 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:32.473286 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e256f-8b64-4342-9c7d-b1bf3b8e68c7 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:32.473509 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e256f-8b64-4342-9c7d-b1bf3b8e68c7 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:32.475425 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 439/866] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:02:32.476216 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e256f-8b64-4342-9c7d-b1bf3b8e68c7 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:32.495417 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 428/867] 10.222.0.60 () {56 vars in 1149 bytes} [Wed Jul 13 23:02:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:32.496775 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a55c0-fcfc-48df-8b2d-e1be6581a54c None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:32.497029 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a55c0-fcfc-48df-8b2d-e1be6581a54c None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:32.497300 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a55c0-fcfc-48df-8b2d-e1be6581a54c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:32.514637 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-335a55c0-fcfc-48df-8b2d-e1be6581a54c None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:32.515860 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-335a55c0-fcfc-48df-8b2d-e1be6581a54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:32.535013 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:32.597620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 429/868] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:02:32.598489 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335a55c0-fcfc-48df-8b2d-e1be6581a54c None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:32.628084 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8e256f-8b64-4342-9c7d-b1bf3b8e68c7 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:32.648984 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:32.673083 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eb5af2-8852-4537-be8a-e8a586e49295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:32.673646 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb5af2-8852-4537-be8a-e8a586e49295 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:32.673918 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb5af2-8852-4537-be8a-e8a586e49295 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:32.674187 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb5af2-8852-4537-be8a-e8a586e49295 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:32.674923 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eb5af2-8852-4537-be8a-e8a586e49295 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:32.676344 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eb5af2-8852-4537-be8a-e8a586e49295 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:32.693130 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 440/869] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Jul 13 23:02:32 2022] POST /identity/v3/services => generated 251 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:02:32.695080 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eb5af2-8852-4537-be8a-e8a586e49295 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:32.778835 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335a55c0-fcfc-48df-8b2d-e1be6581a54c None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:32.847490 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed3f39a-d696-47a4-bcf6-2712bbc3b4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:32.848277 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3f39a-d696-47a4-bcf6-2712bbc3b4f5 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:32.848567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3f39a-d696-47a4-bcf6-2712bbc3b4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:32.848751 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3f39a-d696-47a4-bcf6-2712bbc3b4f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:32.872693 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:32.878005 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eb5af2-8852-4537-be8a-e8a586e49295 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:32.946649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4e795-d705-461e-88b7-bdb64b192634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:32.946649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4e795-d705-461e-88b7-bdb64b192634 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:32.946930 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4e795-d705-461e-88b7-bdb64b192634 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:32.947304 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4e795-d705-461e-88b7-bdb64b192634 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:32.948007 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c4e795-d705-461e-88b7-bdb64b192634 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:32.948553 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed3f39a-d696-47a4-bcf6-2712bbc3b4f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:32.949400 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c4e795-d705-461e-88b7-bdb64b192634 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:32.950976 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed3f39a-d696-47a4-bcf6-2712bbc3b4f5 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:32.964434 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-37c4e795-d705-461e-88b7-bdb64b192634 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 23:02:32.964434 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:32.964434 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 23:02:32.964434 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 23:02:32.964434 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 23:02:32.964434 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 23:02:32.964434 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 23:02:32.964434 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:32.964434 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 23:02:32.964434 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:32.964434 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 23:02:32.964434 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:32.964434 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:32.964434 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:32.964434 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:32.964434 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:32.964434 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:32.965679 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:32.965679 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:32.965679 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:32.965679 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:32.965679 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:32.965679 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:32.965679 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 23:02:32.965679 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 23:02:32.965679 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 23:02:32.965679 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 23:02:32.965679 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:32.965679 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:32.965679 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:32.965679 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:32.965679 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:32.965679 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:32.966690 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:32.966690 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:32.966690 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:32.966690 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:32.966690 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:32.966690 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:32.966690 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:32.966690 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:32.966690 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:32.966690 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:32.966690 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 23:02:32.966690 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:32.966690 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 23:02:32.966690 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:32.969209 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 441/870] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Jul 13 23:02:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:32.975973 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4e795-d705-461e-88b7-bdb64b192634 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:33.019188 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:33.140240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4e795-d705-461e-88b7-bdb64b192634 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:33.155642 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 430/871] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:02:32 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 23:02:33.160321 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed3f39a-d696-47a4-bcf6-2712bbc3b4f5 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:33.200406 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e7a0a-9cc2-4009-be2f-58dd72ca5aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:33.201552 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e7a0a-9cc2-4009-be2f-58dd72ca5aab None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:33.202118 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e7a0a-9cc2-4009-be2f-58dd72ca5aab None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:33.202684 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e7a0a-9cc2-4009-be2f-58dd72ca5aab None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:33.204138 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08e7a0a-9cc2-4009-be2f-58dd72ca5aab None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:33.204734 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08e7a0a-9cc2-4009-be2f-58dd72ca5aab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:33.207418 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08e7a0a-9cc2-4009-be2f-58dd72ca5aab None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:33.216585 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 442/872] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Jul 13 23:02:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:02:33.226310 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e7a0a-9cc2-4009-be2f-58dd72ca5aab None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:33.259610 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:33.321270 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed3f39a-d696-47a4-bcf6-2712bbc3b4f5 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:33.374077 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dc2645-fe80-4e42-95b1-57c0969a6505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:33.374890 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc2645-fe80-4e42-95b1-57c0969a6505 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:33.375172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc2645-fe80-4e42-95b1-57c0969a6505 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:33.375440 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dc2645-fe80-4e42-95b1-57c0969a6505 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:33.391694 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dc2645-fe80-4e42-95b1-57c0969a6505 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:33.392507 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e7a0a-9cc2-4009-be2f-58dd72ca5aab None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:33.393191 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dc2645-fe80-4e42-95b1-57c0969a6505 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:33.394928 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:33.422146 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-76dc2645-fe80-4e42-95b1-57c0969a6505 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:33.422146 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:33.422146 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:33.422146 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:33.422146 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:33.422146 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:33.422146 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:33.422146 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:33.422146 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:33.422146 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:33.422146 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:33.422146 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:33.422146 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:33.422146 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:33.422146 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:33.423142 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:33.423142 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:33.423142 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:33.423142 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:33.423142 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:33.423142 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:33.423142 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:33.423142 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:33.423142 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:33.423142 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:33.423142 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:33.423142 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:33.423142 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:33.423142 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:33.423142 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:33.423142 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:33.423142 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:33.424355 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:33.424355 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:33.424355 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:33.424355 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:33.424355 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 431/873] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:33.433661 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dc2645-fe80-4e42-95b1-57c0969a6505 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:33.439979 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6fe8ae-30f3-4cd4-82f9-4a307a98b3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:33.440990 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6fe8ae-30f3-4cd4-82f9-4a307a98b3df None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:33.441460 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6fe8ae-30f3-4cd4-82f9-4a307a98b3df None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:33.441935 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6fe8ae-30f3-4cd4-82f9-4a307a98b3df None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:33.442850 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6fe8ae-30f3-4cd4-82f9-4a307a98b3df None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:33.444012 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6fe8ae-30f3-4cd4-82f9-4a307a98b3df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:33.445862 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6fe8ae-30f3-4cd4-82f9-4a307a98b3df None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:33.460018 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 443/874] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Jul 13 23:02:33 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:33.796274 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:33.802414 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dc2645-fe80-4e42-95b1-57c0969a6505 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:33.871175 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0191ed-c4f8-421b-95f4-818275f31cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:33.872440 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0191ed-c4f8-421b-95f4-818275f31cbe None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:33.872953 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0191ed-c4f8-421b-95f4-818275f31cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:33.873461 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0191ed-c4f8-421b-95f4-818275f31cbe None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:33.874892 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0191ed-c4f8-421b-95f4-818275f31cbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:33.875456 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0191ed-c4f8-421b-95f4-818275f31cbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:33.878195 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0191ed-c4f8-421b-95f4-818275f31cbe None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:33.901686 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 432/875] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 469 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:33.911060 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6fe8ae-30f3-4cd4-82f9-4a307a98b3df None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:34.109852 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6fe8ae-30f3-4cd4-82f9-4a307a98b3df None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:34.180513 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81467d3a-21ad-47d9-bd82-a1b23d195d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:34.182428 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81467d3a-21ad-47d9-bd82-a1b23d195d61 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:34.183229 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81467d3a-21ad-47d9-bd82-a1b23d195d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:34.184041 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81467d3a-21ad-47d9-bd82-a1b23d195d61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:34.198736 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81467d3a-21ad-47d9-bd82-a1b23d195d61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:34.202045 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81467d3a-21ad-47d9-bd82-a1b23d195d61 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:34.232760 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-81467d3a-21ad-47d9-bd82-a1b23d195d61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:34.232760 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:34.232760 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:34.232760 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:34.232760 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:34.232760 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:34.232760 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:34.232760 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:34.232760 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:34.232760 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:34.232760 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:34.232760 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:34.232760 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 23:02:34.232760 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 23:02:34.232760 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 23:02:34.234522 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 23:02:34.234522 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:34.234522 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:34.234522 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:34.234522 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:34.234522 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:34.234522 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:34.234522 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:34.234522 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:34.234522 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:34.234522 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:34.234522 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:34.234522 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:34.234522 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:34.234522 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:34.234522 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:34.234522 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:34.236342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 23:02:34.236342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 23:02:34.236342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 23:02:34.236342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 23:02:34.236342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 23:02:34.236342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 23:02:34.236342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:34.236342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:34.241082 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 444/876] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:02:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:34.251692 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0191ed-c4f8-421b-95f4-818275f31cbe None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:34.369977 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0191ed-c4f8-421b-95f4-818275f31cbe None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:34.407125 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b382f0f6-180b-4d3e-963f-50a1ae5b2772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:34.407844 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382f0f6-180b-4d3e-963f-50a1ae5b2772 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:34.408391 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382f0f6-180b-4d3e-963f-50a1ae5b2772 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:34.408742 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382f0f6-180b-4d3e-963f-50a1ae5b2772 None admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:34.409622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b382f0f6-180b-4d3e-963f-50a1ae5b2772 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:34.410004 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b382f0f6-180b-4d3e-963f-50a1ae5b2772 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:34.411557 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b382f0f6-180b-4d3e-963f-50a1ae5b2772 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:34.434109 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 433/877] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jul 13 23:02:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:34.440063 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81467d3a-21ad-47d9-bd82-a1b23d195d61 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:34.476040 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:34.578004 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81467d3a-21ad-47d9-bd82-a1b23d195d61 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:34.618170 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db33bfd-deb9-439b-8440-4481fcf5e593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:34.618771 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db33bfd-deb9-439b-8440-4481fcf5e593 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:34.619059 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db33bfd-deb9-439b-8440-4481fcf5e593 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:34.619331 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db33bfd-deb9-439b-8440-4481fcf5e593 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:34.620255 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db33bfd-deb9-439b-8440-4481fcf5e593 None admin] RBAC: Adding query filter params (role.id=92417b14d4984ed9956dcd14f97564f3, scope.project.id=ab0a5581e4e444c293815e1304e20c05, user.id=a32702a67a7b49ffa153fc59251fd169) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:34.620564 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db33bfd-deb9-439b-8440-4481fcf5e593 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:34.622092 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db33bfd-deb9-439b-8440-4481fcf5e593 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:34.629587 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 445/878] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:02:34 2022] GET /identity/v3/role_assignments?user.id=a32702a67a7b49ffa153fc59251fd169&scope.project.id=ab0a5581e4e444c293815e1304e20c05&role.id=92417b14d4984ed9956dcd14f97564f3 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:34.644562 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52ffbf-d557-4e29-bec6-17ce30eb2e57 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:34.645040 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52ffbf-d557-4e29-bec6-17ce30eb2e57 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:34.645504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52ffbf-d557-4e29-bec6-17ce30eb2e57 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:34.649189 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 434/879] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:34.658311 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6902344-5ee4-4c68-aebd-e94e5b84fa8c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:34.658768 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6902344-5ee4-4c68-aebd-e94e5b84fa8c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:34.659240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6902344-5ee4-4c68-aebd-e94e5b84fa8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:34.688281 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-a6902344-5ee4-4c68-aebd-e94e5b84fa8c None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:34.690327 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a6902344-5ee4-4c68-aebd-e94e5b84fa8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:34.817709 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 446/880] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:34 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:02:34.858015 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e52ffbf-d557-4e29-bec6-17ce30eb2e57 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:34.902852 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:35.056126 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e52ffbf-d557-4e29-bec6-17ce30eb2e57 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:35.125504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ea42fc-26cc-4a32-be19-77e40921fded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:35.126652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea42fc-26cc-4a32-be19-77e40921fded None admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:35.127135 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea42fc-26cc-4a32-be19-77e40921fded None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:35.127648 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea42fc-26cc-4a32-be19-77e40921fded None admin] PATH_INFO: `/v3/services` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:35.129003 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ea42fc-26cc-4a32-be19-77e40921fded None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:35.132091 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ea42fc-26cc-4a32-be19-77e40921fded None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:35.156575 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 435/881] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Jul 13 23:02:34 2022] POST /identity/v3/services => generated 269 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:02:35.506391 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec2fe9-481a-4b04-8ecf-e52f2a857ae8 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:35.506890 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec2fe9-481a-4b04-8ecf-e52f2a857ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:35.506890 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec2fe9-481a-4b04-8ecf-e52f2a857ae8 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:35.508533 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 447/882] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:35.514291 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291895f6-26bd-451a-b8ef-3d430b4fd3c1 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:35.514932 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291895f6-26bd-451a-b8ef-3d430b4fd3c1 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:35.515416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291895f6-26bd-451a-b8ef-3d430b4fd3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:35.533392 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-291895f6-26bd-451a-b8ef-3d430b4fd3c1 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:35.535017 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-291895f6-26bd-451a-b8ef-3d430b4fd3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:35.620143 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 436/883] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 23:02:35.643029 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec2fe9-481a-4b04-8ecf-e52f2a857ae8 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:35.667404 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:35.751237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec2fe9-481a-4b04-8ecf-e52f2a857ae8 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:35.788946 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb07dbd-40a8-4488-a191-6ac288210520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:35.789676 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb07dbd-40a8-4488-a191-6ac288210520 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:35.790003 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb07dbd-40a8-4488-a191-6ac288210520 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:35.790284 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb07dbd-40a8-4488-a191-6ac288210520 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:35.790948 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb07dbd-40a8-4488-a191-6ac288210520 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:35.792350 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb07dbd-40a8-4488-a191-6ac288210520 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:35.805154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-fbb07dbd-40a8-4488-a191-6ac288210520 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 23:02:35.805154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:35.805154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 23:02:35.805154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 23:02:35.805154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 23:02:35.805154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 23:02:35.805154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 23:02:35.805154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:35.805154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 23:02:35.805154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:35.805154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 23:02:35.805154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:35.805154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:35.805154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:35.805154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:35.805154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:35.805154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:35.806193 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:35.806193 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:35.806193 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:35.806193 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:35.806193 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:35.806193 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:35.806193 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 23:02:35.806193 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 23:02:35.806193 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 23:02:35.806193 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 23:02:35.806193 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:35.806193 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:35.806193 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:35.806193 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:35.806193 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:35.806193 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:35.807280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:35.807280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:35.807280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:35.807280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:35.807280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:35.807280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:35.807280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:35.807280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:35.807280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:35.807280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:35.807280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 23:02:35.807280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:35.807280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 23:02:35.807280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:35.807280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 448/884] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jul 13 23:02:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:35.816909 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291895f6-26bd-451a-b8ef-3d430b4fd3c1 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:35.825559 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:35.848476 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba2b42-fbdc-4128-9bc4-7ca809767bc1 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:35.848476 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba2b42-fbdc-4128-9bc4-7ca809767bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:35.848759 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba2b42-fbdc-4128-9bc4-7ca809767bc1 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:35.851379 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 449/885] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:35.857129 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d157bb6-f5b3-4fad-bf8c-5bb3a4da441a None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:35.857446 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d157bb6-f5b3-4fad-bf8c-5bb3a4da441a None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:35.857681 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d157bb6-f5b3-4fad-bf8c-5bb3a4da441a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:35.874871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-0d157bb6-f5b3-4fad-bf8c-5bb3a4da441a None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:35.877424 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0d157bb6-f5b3-4fad-bf8c-5bb3a4da441a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:35.932786 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291895f6-26bd-451a-b8ef-3d430b4fd3c1 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:35.951979 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 450/886] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 23:02:35.972384 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c2a272-cb0f-415f-abd2-2144ca21b204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:35.973372 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2a272-cb0f-415f-abd2-2144ca21b204 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:35.973873 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2a272-cb0f-415f-abd2-2144ca21b204 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:35.974297 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2a272-cb0f-415f-abd2-2144ca21b204 None admin] PATH_INFO: `/v3/services` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:35.976281 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c2a272-cb0f-415f-abd2-2144ca21b204 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:35.976624 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d157bb6-f5b3-4fad-bf8c-5bb3a4da441a None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:35.976914 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c2a272-cb0f-415f-abd2-2144ca21b204 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:35.979293 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c2a272-cb0f-415f-abd2-2144ca21b204 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:35.989793 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 437/887] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jul 13 23:02:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:35.991398 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c2a272-cb0f-415f-abd2-2144ca21b204 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:36.004148 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:36.066621 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:36.092061 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d157bb6-f5b3-4fad-bf8c-5bb3a4da441a None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:36.101773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c2a272-cb0f-415f-abd2-2144ca21b204 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:36.130060 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80682a10-7860-40d1-baf2-056ddad42dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:36.131419 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80682a10-7860-40d1-baf2-056ddad42dd1 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:36.131832 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80682a10-7860-40d1-baf2-056ddad42dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:36.132200 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80682a10-7860-40d1-baf2-056ddad42dd1 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:36.133142 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80682a10-7860-40d1-baf2-056ddad42dd1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:36.134724 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80682a10-7860-40d1-baf2-056ddad42dd1 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:36.138847 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6448e73b-3fd3-4986-a6cb-7383a986616f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:36.139883 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448e73b-3fd3-4986-a6cb-7383a986616f None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:36.140170 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448e73b-3fd3-4986-a6cb-7383a986616f None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:36.140578 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6448e73b-3fd3-4986-a6cb-7383a986616f None admin] PATH_INFO: `/v3/services` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:36.141198 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6448e73b-3fd3-4986-a6cb-7383a986616f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:36.141533 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6448e73b-3fd3-4986-a6cb-7383a986616f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:36.142876 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6448e73b-3fd3-4986-a6cb-7383a986616f None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:36.155840 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 438/889] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jul 13 23:02:35 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:02:36.155840 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6448e73b-3fd3-4986-a6cb-7383a986616f None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:36.156368 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 451/889] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Jul 13 23:02:35 2022] POST /identity/v3/services => generated 241 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:02:36.274078 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6448e73b-3fd3-4986-a6cb-7383a986616f None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:36.313860 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:36.321496 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fc513-0352-4f86-870d-c1bc3de6a57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:36.321496 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fc513-0352-4f86-870d-c1bc3de6a57a None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:36.321496 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fc513-0352-4f86-870d-c1bc3de6a57a None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:36.321820 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fc513-0352-4f86-870d-c1bc3de6a57a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:36.322398 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045fc513-0352-4f86-870d-c1bc3de6a57a None admin] RBAC: Adding query filter params (interface=public, service_id=88ed53b9dab84818a59b4dc49fa91793, region_id=RegionOne) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:36.322704 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045fc513-0352-4f86-870d-c1bc3de6a57a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:36.324191 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045fc513-0352-4f86-870d-c1bc3de6a57a None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:36.331689 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 439/890] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Jul 13 23:02:36 2022] GET /identity/v3/endpoints?service_id=88ed53b9dab84818a59b4dc49fa91793&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:36.336546 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80682a10-7860-40d1-baf2-056ddad42dd1 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:36.434711 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:36.481592 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80682a10-7860-40d1-baf2-056ddad42dd1 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:36.543327 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee278c9-e0cb-4745-8ae8-a78110fbfdfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:36.544517 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee278c9-e0cb-4745-8ae8-a78110fbfdfe None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:36.544927 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee278c9-e0cb-4745-8ae8-a78110fbfdfe None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:36.545405 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee278c9-e0cb-4745-8ae8-a78110fbfdfe None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:36.546682 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee278c9-e0cb-4745-8ae8-a78110fbfdfe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:36.549261 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee278c9-e0cb-4745-8ae8-a78110fbfdfe None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:36.567733 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 452/891] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Jul 13 23:02:36 2022] GET /identity/v3/services => generated 1497 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:02:36.899805 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349e78c-3cd2-4430-96ad-a3466c58bc19 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:36.900094 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349e78c-3cd2-4430-96ad-a3466c58bc19 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:36.900480 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349e78c-3cd2-4430-96ad-a3466c58bc19 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:36.902048 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 440/892] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:02:36.907494 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fffc31-527d-4030-9a19-d11236043871 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:36.907873 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fffc31-527d-4030-9a19-d11236043871 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:36.908128 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fffc31-527d-4030-9a19-d11236043871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:36.926348 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-32fffc31-527d-4030-9a19-d11236043871 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:36.927681 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-32fffc31-527d-4030-9a19-d11236043871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:37.002710 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 453/893] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 23:02:37.026236 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2349e78c-3cd2-4430-96ad-a3466c58bc19 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:37.185662 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2349e78c-3cd2-4430-96ad-a3466c58bc19 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:37.248952 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf90cb-18ea-473e-96ed-fe2e466ea0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:37.250319 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf90cb-18ea-473e-96ed-fe2e466ea0e0 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:37.250801 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf90cb-18ea-473e-96ed-fe2e466ea0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:37.251275 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf90cb-18ea-473e-96ed-fe2e466ea0e0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:37.271031 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cf90cb-18ea-473e-96ed-fe2e466ea0e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:02:37.288421 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-13cf90cb-18ea-473e-96ed-fe2e466ea0e0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 23:02:37.288421 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:37.288421 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:37.288421 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:37.288421 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:37.288421 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:37.288421 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:37.288421 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:37.288421 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:37.288421 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:37.288421 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:37.288421 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:37.288421 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:02:37.288421 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:02:37.288421 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:02:37.288421 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:02:37.291714 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:02:37.291714 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:02:37.291714 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:02:37.291714 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:02:37.291714 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:37.291714 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:37.291714 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:37.291714 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:37.291714 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:37.291714 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:37.291714 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:37.291714 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:37.291714 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:37.291714 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:37.291714 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:37.291714 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:37.293304 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:37.293304 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:37.293304 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:37.293304 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:37.293304 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:02:37.293304 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:02:37.293304 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:02:37.293304 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:02:37.293304 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:02:37.293304 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:02:37.293304 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 23:02:37.293304 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:37.293304 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 441/894] 10.222.0.60 () {56 vars in 1120 bytes} [Wed Jul 13 23:02:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:37.297315 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fffc31-527d-4030-9a19-d11236043871 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:37.412676 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fffc31-527d-4030-9a19-d11236043871 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:37.416263 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:37.451527 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1f55c-ac02-45e3-85c3-ef9fae821248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:37.452152 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1f55c-ac02-45e3-85c3-ef9fae821248 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:37.452435 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1f55c-ac02-45e3-85c3-ef9fae821248 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:37.452712 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1f55c-ac02-45e3-85c3-ef9fae821248 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:37.453477 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca1f55c-ac02-45e3-85c3-ef9fae821248 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:37.453784 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca1f55c-ac02-45e3-85c3-ef9fae821248 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:37.455187 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca1f55c-ac02-45e3-85c3-ef9fae821248 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:37.465812 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 454/895] 10.222.0.60 () {56 vars in 1115 bytes} [Wed Jul 13 23:02:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:37.472760 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf90cb-18ea-473e-96ed-fe2e466ea0e0 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:37.537202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:37.591148 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf90cb-18ea-473e-96ed-fe2e466ea0e0 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:37.629230 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4951bc-8cff-47e4-bdc1-1f6ce1127c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:37.630042 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4951bc-8cff-47e4-bdc1-1f6ce1127c6e None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:37.630325 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4951bc-8cff-47e4-bdc1-1f6ce1127c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:37.630600 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4951bc-8cff-47e4-bdc1-1f6ce1127c6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:37.643595 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4951bc-8cff-47e4-bdc1-1f6ce1127c6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:37.645093 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4951bc-8cff-47e4-bdc1-1f6ce1127c6e None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:37.664761 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-cc4951bc-8cff-47e4-bdc1-1f6ce1127c6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:37.664761 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:37.664761 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:37.664761 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:37.664761 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:37.664761 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:37.664761 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:37.664761 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:37.664761 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:37.664761 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:37.664761 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:37.664761 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:37.664761 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:37.664761 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:37.664761 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:37.668092 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:37.668092 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:37.668092 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:37.668092 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:37.668092 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:37.668092 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:37.668092 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:37.668092 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:37.668092 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:37.668092 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:37.668092 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:37.668092 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:37.668092 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:37.668092 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:37.668092 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:37.668092 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:37.668092 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:37.671500 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:37.671500 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:37.671500 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:37.671500 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:37.671500 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 442/896] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:37.672551 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1f55c-ac02-45e3-85c3-ef9fae821248 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:37.814888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1f55c-ac02-45e3-85c3-ef9fae821248 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:37.874823 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:37.888738 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2428d2b-1dba-4ec9-8363-6cc60d254f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:37.889937 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2428d2b-1dba-4ec9-8363-6cc60d254f71 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:37.890677 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2428d2b-1dba-4ec9-8363-6cc60d254f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:37.891172 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2428d2b-1dba-4ec9-8363-6cc60d254f71 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:37.892609 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2428d2b-1dba-4ec9-8363-6cc60d254f71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:37.893161 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2428d2b-1dba-4ec9-8363-6cc60d254f71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:37.895795 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2428d2b-1dba-4ec9-8363-6cc60d254f71 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:37.919689 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 455/897] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:37.925241 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4951bc-8cff-47e4-bdc1-1f6ce1127c6e None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:38.050631 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4951bc-8cff-47e4-bdc1-1f6ce1127c6e None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:38.059577 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:38.092283 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b1a94-757f-4878-b174-e0e624e0bab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:38.093007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b1a94-757f-4878-b174-e0e624e0bab2 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:38.093471 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b1a94-757f-4878-b174-e0e624e0bab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:38.093753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b1a94-757f-4878-b174-e0e624e0bab2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:38.102095 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9b1a94-757f-4878-b174-e0e624e0bab2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:38.103635 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9b1a94-757f-4878-b174-e0e624e0bab2 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:38.121318 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-2f9b1a94-757f-4878-b174-e0e624e0bab2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 23:02:38.121318 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:38.121318 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:38.121318 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:38.121318 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:38.121318 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:38.121318 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:38.121318 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:38.121318 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:38.121318 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:38.121318 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:38.121318 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:38.121318 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 23:02:38.121318 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 23:02:38.121318 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 23:02:38.121318 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 23:02:38.124753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:38.124753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:38.124753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 23:02:38.124753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:38.124753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 23:02:38.124753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:38.124753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:38.124753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:38.124753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:38.124753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:38.124753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:38.124753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:38.124753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:38.124753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:38.124753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:38.124753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:38.124753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:38.124753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:38.128226 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:38.128226 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:38.128226 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 23:02:38.128226 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 23:02:38.128226 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 23:02:38.128226 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 23:02:38.128226 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 23:02:38.128226 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 23:02:38.128226 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 23:02:38.128226 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:38.128226 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 443/898] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Jul 13 23:02:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:38.130232 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2428d2b-1dba-4ec9-8363-6cc60d254f71 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:38.304216 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2428d2b-1dba-4ec9-8363-6cc60d254f71 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:38.379459 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80f2db-b52c-4635-af2e-22ee556dacec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:38.380510 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80f2db-b52c-4635-af2e-22ee556dacec None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:38.381049 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80f2db-b52c-4635-af2e-22ee556dacec None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:38.381639 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80f2db-b52c-4635-af2e-22ee556dacec None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:38.383517 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a80f2db-b52c-4635-af2e-22ee556dacec None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:38.384381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a80f2db-b52c-4635-af2e-22ee556dacec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:38.387074 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a80f2db-b52c-4635-af2e-22ee556dacec None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:38.422758 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 456/899] 10.222.0.60 () {56 vars in 1149 bytes} [Wed Jul 13 23:02:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:38.428664 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b1a94-757f-4878-b174-e0e624e0bab2 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:38.534012 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22beea2-6e70-4efc-a37c-6884dd1bcc7c None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:38.534529 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22beea2-6e70-4efc-a37c-6884dd1bcc7c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:38.535038 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22beea2-6e70-4efc-a37c-6884dd1bcc7c None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:38.540079 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b1a94-757f-4878-b174-e0e624e0bab2 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:38.543943 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 457/900] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:38.547867 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25957589-6cb3-4897-89f5-fd3733a3438d None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:38.548521 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25957589-6cb3-4897-89f5-fd3733a3438d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:38.549020 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25957589-6cb3-4897-89f5-fd3733a3438d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:38.578993 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a914efe-bcb7-4115-a079-c13768f3d620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:38.579701 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a914efe-bcb7-4115-a079-c13768f3d620 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:38.579993 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a914efe-bcb7-4115-a079-c13768f3d620 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:38.580266 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a914efe-bcb7-4115-a079-c13768f3d620 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:38.581257 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-25957589-6cb3-4897-89f5-fd3733a3438d None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:38.583491 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-25957589-6cb3-4897-89f5-fd3733a3438d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:38.618158 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:38.657207 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a914efe-bcb7-4115-a079-c13768f3d620 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:38.658603 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a914efe-bcb7-4115-a079-c13768f3d620 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:38.678036 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:38.740471 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 458/901] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 23:02:38.776875 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25957589-6cb3-4897-89f5-fd3733a3438d None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:38.778070 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 444/902] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:02:38 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 23:02:38.784652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a914efe-bcb7-4115-a079-c13768f3d620 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:38.868529 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:38.907711 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a914efe-bcb7-4115-a079-c13768f3d620 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:38.947332 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e3bf32-2468-4581-a1b4-9cb32753c5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:38.948419 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3bf32-2468-4581-a1b4-9cb32753c5df None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:38.948855 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3bf32-2468-4581-a1b4-9cb32753c5df None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:38.949307 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3bf32-2468-4581-a1b4-9cb32753c5df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:38.962541 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e3bf32-2468-4581-a1b4-9cb32753c5df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:38.964315 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e3bf32-2468-4581-a1b4-9cb32753c5df None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:38.985040 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-27e3bf32-2468-4581-a1b4-9cb32753c5df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:38.985040 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:38.985040 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:38.985040 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:38.985040 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:38.985040 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:38.985040 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:38.985040 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:38.985040 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:38.985040 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:38.985040 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:38.985040 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:38.985040 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:38.985040 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:38.985040 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:38.986089 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:38.986089 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:38.986089 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:38.986089 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:38.986089 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:38.986089 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:38.986089 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:38.986089 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:38.986089 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:38.986089 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:38.986089 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:38.986089 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:38.986089 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:38.986089 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:38.986089 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:38.986089 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:38.986089 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:38.986973 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:38.986973 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:38.986973 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:38.986973 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:38.990607 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25957589-6cb3-4897-89f5-fd3733a3438d None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:38.993578 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 445/903] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:02:38.994780 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e3bf32-2468-4581-a1b4-9cb32753c5df None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:39.060216 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec833bc-91f2-427d-9d17-c7dadf2fce55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:39.062286 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec833bc-91f2-427d-9d17-c7dadf2fce55 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:39.063159 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec833bc-91f2-427d-9d17-c7dadf2fce55 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:39.063958 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec833bc-91f2-427d-9d17-c7dadf2fce55 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:39.065766 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec833bc-91f2-427d-9d17-c7dadf2fce55 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:39.069833 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec833bc-91f2-427d-9d17-c7dadf2fce55 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:39.097982 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-bec833bc-91f2-427d-9d17-c7dadf2fce55 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 23:02:39.097982 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:39.097982 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 23:02:39.097982 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 23:02:39.097982 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 23:02:39.097982 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 23:02:39.097982 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 23:02:39.097982 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:39.097982 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 23:02:39.097982 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:39.097982 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 23:02:39.097982 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:39.097982 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:39.097982 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:39.097982 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:39.097982 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:39.099944 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:39.099944 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:39.099944 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:39.099944 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:39.099944 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:39.099944 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:39.099944 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:39.099944 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 23:02:39.099944 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 23:02:39.099944 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 23:02:39.099944 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 23:02:39.099944 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:39.099944 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:39.099944 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:39.099944 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:39.099944 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:39.101613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:39.101613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:39.101613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:39.101613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:39.101613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:39.101613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:39.101613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:39.101613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:39.101613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:39.101613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:39.101613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:39.101613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 23:02:39.101613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:39.101613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 23:02:39.101613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:39.112357 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e3bf32-2468-4581-a1b4-9cb32753c5df None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:39.113441 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 459/904] 10.222.0.60 () {56 vars in 1132 bytes} [Wed Jul 13 23:02:38 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:39.118830 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec833bc-91f2-427d-9d17-c7dadf2fce55 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:39.129821 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:39.150484 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b1f18-e20b-427d-9765-38e1c068a0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:39.151175 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b1f18-e20b-427d-9765-38e1c068a0dd None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:39.151439 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b1f18-e20b-427d-9765-38e1c068a0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:39.151767 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b1f18-e20b-427d-9765-38e1c068a0dd None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:39.152484 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2b1f18-e20b-427d-9765-38e1c068a0dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:39.152791 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2b1f18-e20b-427d-9765-38e1c068a0dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:39.155430 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2b1f18-e20b-427d-9765-38e1c068a0dd None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:39.169540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 446/905] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:02:39.175280 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b1f18-e20b-427d-9765-38e1c068a0dd None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:39.214136 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:39.281693 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec833bc-91f2-427d-9d17-c7dadf2fce55 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:39.296478 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b1f18-e20b-427d-9765-38e1c068a0dd None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:39.319595 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed364801-3cec-4a4c-baca-dc81082ea0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:39.320946 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed364801-3cec-4a4c-baca-dc81082ea0e4 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:39.321436 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed364801-3cec-4a4c-baca-dc81082ea0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:39.321925 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed364801-3cec-4a4c-baca-dc81082ea0e4 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:39.322862 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed364801-3cec-4a4c-baca-dc81082ea0e4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:39.323378 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed364801-3cec-4a4c-baca-dc81082ea0e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:39.325020 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed364801-3cec-4a4c-baca-dc81082ea0e4 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:39.336124 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 460/906] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Jul 13 23:02:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:39.337478 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14285e92-c2d0-46e8-b43c-2e3af9ad41ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:39.337872 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed364801-3cec-4a4c-baca-dc81082ea0e4 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:39.338297 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14285e92-c2d0-46e8-b43c-2e3af9ad41ba None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:39.338603 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14285e92-c2d0-46e8-b43c-2e3af9ad41ba None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:39.338882 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14285e92-c2d0-46e8-b43c-2e3af9ad41ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:39.346845 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14285e92-c2d0-46e8-b43c-2e3af9ad41ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:39.348746 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14285e92-c2d0-46e8-b43c-2e3af9ad41ba None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:39.364345 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-14285e92-c2d0-46e8-b43c-2e3af9ad41ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:39.364345 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:39.364345 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:39.364345 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:39.364345 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:39.364345 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:39.364345 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:39.364345 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:39.364345 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:39.364345 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:39.364345 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:39.364345 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:39.364345 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 23:02:39.364345 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 23:02:39.364345 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 23:02:39.365881 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 23:02:39.365881 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:39.365881 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:39.365881 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:39.365881 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:39.365881 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:39.365881 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:39.365881 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:39.365881 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:39.365881 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:39.365881 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:39.365881 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:39.365881 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:39.365881 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:39.365881 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:39.365881 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:39.365881 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:39.366775 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 23:02:39.366775 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 23:02:39.366775 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 23:02:39.366775 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 23:02:39.366775 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 23:02:39.366775 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 23:02:39.366775 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:39.366775 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:39.366775 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 447/907] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:02:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:02:39.372420 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217428e8-85d5-4f0c-8d1b-f402a22e0e1b None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:39.372710 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217428e8-85d5-4f0c-8d1b-f402a22e0e1b None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:39.372986 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217428e8-85d5-4f0c-8d1b-f402a22e0e1b None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:39.374512 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 448/908] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:39.375077 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217428e8-85d5-4f0c-8d1b-f402a22e0e1b None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:39.435451 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:39.487399 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed364801-3cec-4a4c-baca-dc81082ea0e4 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:39.496178 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:39.514343 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217428e8-85d5-4f0c-8d1b-f402a22e0e1b None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:39.528560 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2e25e2-372d-434a-95be-597e1260a9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:39.529230 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e25e2-372d-434a-95be-597e1260a9c9 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:39.529509 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e25e2-372d-434a-95be-597e1260a9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:39.529920 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e25e2-372d-434a-95be-597e1260a9c9 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:39.530733 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2e25e2-372d-434a-95be-597e1260a9c9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:39.531046 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2e25e2-372d-434a-95be-597e1260a9c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:39.532487 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2e25e2-372d-434a-95be-597e1260a9c9 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:39.539292 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 461/909] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Jul 13 23:02:39 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:39.540409 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c9be2-d898-4bbb-bf49-4e3e05445eeb None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:39.540700 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c9be2-d898-4bbb-bf49-4e3e05445eeb None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:39.540976 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c9be2-d898-4bbb-bf49-4e3e05445eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:39.560378 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-6c4c9be2-d898-4bbb-bf49-4e3e05445eeb None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:39.561598 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6c4c9be2-d898-4bbb-bf49-4e3e05445eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:39.563275 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4271ba83-b35a-414b-9500-0448ae5adb60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:39.566006 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271ba83-b35a-414b-9500-0448ae5adb60 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:39.566522 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271ba83-b35a-414b-9500-0448ae5adb60 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:39.567014 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271ba83-b35a-414b-9500-0448ae5adb60 None admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:39.569220 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4271ba83-b35a-414b-9500-0448ae5adb60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:39.569773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4271ba83-b35a-414b-9500-0448ae5adb60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:39.574027 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4271ba83-b35a-414b-9500-0448ae5adb60 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:39.601993 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 449/910] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jul 13 23:02:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:39.601993 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4271ba83-b35a-414b-9500-0448ae5adb60 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:39.648133 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 462/911] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:02:39.648923 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c9be2-d898-4bbb-bf49-4e3e05445eeb None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:39.740051 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:39.744665 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4271ba83-b35a-414b-9500-0448ae5adb60 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:39.782845 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88831b39-31e6-49b6-bef4-5b9e3dcee38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:39.783525 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88831b39-31e6-49b6-bef4-5b9e3dcee38a None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:39.783842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88831b39-31e6-49b6-bef4-5b9e3dcee38a None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:39.784119 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88831b39-31e6-49b6-bef4-5b9e3dcee38a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:39.784951 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88831b39-31e6-49b6-bef4-5b9e3dcee38a None admin] RBAC: Adding query filter params (interface=public, service_id=1609e870a72c4bda9f123d3aa2779d35, region_id=RegionOne) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:39.785264 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88831b39-31e6-49b6-bef4-5b9e3dcee38a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:39.786702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88831b39-31e6-49b6-bef4-5b9e3dcee38a None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:39.793087 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 450/912] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Jul 13 23:02:39 2022] GET /identity/v3/endpoints?service_id=1609e870a72c4bda9f123d3aa2779d35&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:39.793974 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88831b39-31e6-49b6-bef4-5b9e3dcee38a None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:39.804883 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:39.809332 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c9be2-d898-4bbb-bf49-4e3e05445eeb None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:39.872089 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e443093b-d25e-41a6-b3c2-0a2d503d48dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:39.873228 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443093b-d25e-41a6-b3c2-0a2d503d48dd None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:39.873700 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443093b-d25e-41a6-b3c2-0a2d503d48dd None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:39.874153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443093b-d25e-41a6-b3c2-0a2d503d48dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:39.875824 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e443093b-d25e-41a6-b3c2-0a2d503d48dd None admin] RBAC: Adding query filter params (role.id=f7411bbcd41a4f959f76b130485bfb89, scope.project.id=ab0a5581e4e444c293815e1304e20c05, user.id=a32702a67a7b49ffa153fc59251fd169) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:39.876397 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e443093b-d25e-41a6-b3c2-0a2d503d48dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:39.879262 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e443093b-d25e-41a6-b3c2-0a2d503d48dd None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:39.890391 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 463/913] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:02:39 2022] GET /identity/v3/role_assignments?user.id=a32702a67a7b49ffa153fc59251fd169&scope.project.id=ab0a5581e4e444c293815e1304e20c05&role.id=f7411bbcd41a4f959f76b130485bfb89 => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:39.892890 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48bfa2-3bfa-4e0c-a78a-c4527eaa9e61 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:39.893369 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48bfa2-3bfa-4e0c-a78a-c4527eaa9e61 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:39.893855 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48bfa2-3bfa-4e0c-a78a-c4527eaa9e61 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:39.896877 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 464/914] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:39.898366 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da48bfa2-3bfa-4e0c-a78a-c4527eaa9e61 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:39.941180 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88831b39-31e6-49b6-bef4-5b9e3dcee38a None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:40.009647 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd630f-56f6-49c5-86ab-34493f10bb76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:40.011017 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd630f-56f6-49c5-86ab-34493f10bb76 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:40.011525 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd630f-56f6-49c5-86ab-34493f10bb76 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:40.012076 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd630f-56f6-49c5-86ab-34493f10bb76 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:40.013367 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedd630f-56f6-49c5-86ab-34493f10bb76 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:40.016148 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedd630f-56f6-49c5-86ab-34493f10bb76 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:40.040319 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-bedd630f-56f6-49c5-86ab-34493f10bb76 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 23:02:40.040319 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:40.040319 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 23:02:40.040319 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 23:02:40.040319 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 23:02:40.040319 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 23:02:40.040319 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 23:02:40.040319 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:40.040319 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 23:02:40.040319 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:40.040319 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 23:02:40.040319 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:40.040319 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:40.040319 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:40.040319 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:40.040319 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:40.040319 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:40.041397 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:40.041397 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:40.041397 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:40.041397 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:40.041397 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:40.041397 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:40.041397 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 23:02:40.041397 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 23:02:40.041397 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 23:02:40.041397 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 23:02:40.041397 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:40.041397 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:40.041397 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:40.041397 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:40.041397 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:40.041397 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:40.042233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:40.042233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:40.042233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:40.042233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:40.042233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:40.042233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:40.042233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:40.042233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:40.042233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:40.042233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:40.042233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 23:02:40.042233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:40.042233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 23:02:40.042233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:40.042972 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 451/915] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jul 13 23:02:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:40.045141 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09036b51-4b05-4138-b2e8-23b9ac3e66fe None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:40.045652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09036b51-4b05-4138-b2e8-23b9ac3e66fe None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:40.046163 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09036b51-4b05-4138-b2e8-23b9ac3e66fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:40.061222 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da48bfa2-3bfa-4e0c-a78a-c4527eaa9e61 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:40.076049 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-09036b51-4b05-4138-b2e8-23b9ac3e66fe None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:40.078112 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-09036b51-4b05-4138-b2e8-23b9ac3e66fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:40.111411 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb05d6aa-994f-4130-9054-db6c0ee868f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:40.113247 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05d6aa-994f-4130-9054-db6c0ee868f2 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:40.113765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05d6aa-994f-4130-9054-db6c0ee868f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:40.114270 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05d6aa-994f-4130-9054-db6c0ee868f2 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:40.116184 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb05d6aa-994f-4130-9054-db6c0ee868f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:40.118389 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:40.118886 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb05d6aa-994f-4130-9054-db6c0ee868f2 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:40.139574 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 465/916] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Jul 13 23:02:39 2022] GET /identity/v3/services => generated 1497 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:02:40.139574 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb05d6aa-994f-4130-9054-db6c0ee868f2 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:40.192328 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:40.249206 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 452/917] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:02:40.273797 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09036b51-4b05-4138-b2e8-23b9ac3e66fe None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:40.364340 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb05d6aa-994f-4130-9054-db6c0ee868f2 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:40.421700 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3eefa-8627-43f2-bb37-da4661949c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:40.422754 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3eefa-8627-43f2-bb37-da4661949c13 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:40.423266 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3eefa-8627-43f2-bb37-da4661949c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:40.423814 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3eefa-8627-43f2-bb37-da4661949c13 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:40.424745 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee3eefa-8627-43f2-bb37-da4661949c13 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:40.425270 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee3eefa-8627-43f2-bb37-da4661949c13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:40.426878 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee3eefa-8627-43f2-bb37-da4661949c13 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:40.433801 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 466/918] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jul 13 23:02:40 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:40.442058 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3eefa-8627-43f2-bb37-da4661949c13 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:40.476191 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09036b51-4b05-4138-b2e8-23b9ac3e66fe None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:40.518715 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ff446-e296-47c5-af16-5733c7841a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:40.519991 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ff446-e296-47c5-af16-5733c7841a18 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:40.520543 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ff446-e296-47c5-af16-5733c7841a18 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:40.521017 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ff446-e296-47c5-af16-5733c7841a18 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:40.521973 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881ff446-e296-47c5-af16-5733c7841a18 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:40.523862 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881ff446-e296-47c5-af16-5733c7841a18 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:40.539622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-881ff446-e296-47c5-af16-5733c7841a18 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 23:02:40.539622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:40.539622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 23:02:40.539622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 23:02:40.539622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 23:02:40.539622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 23:02:40.539622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 23:02:40.539622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:40.539622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 23:02:40.539622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:40.539622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 23:02:40.539622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:40.539622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:40.539622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:40.539622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:40.539622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:40.539622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:40.540688 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:40.540688 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:40.540688 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:40.540688 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:40.540688 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:40.540688 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:40.540688 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 23:02:40.540688 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 23:02:40.540688 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 23:02:40.540688 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 23:02:40.540688 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:40.540688 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:40.540688 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:40.540688 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:40.540688 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:40.540688 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:40.541540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:40.541540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:40.541540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:40.541540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:40.541540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:40.541540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:40.541540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:40.541540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:40.541540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:40.541540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:40.541540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 23:02:40.541540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:40.541540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 23:02:40.541540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:40.549408 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 453/919] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Jul 13 23:02:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:40.551189 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ff446-e296-47c5-af16-5733c7841a18 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:40.582957 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:40.633851 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:40.655825 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3eefa-8627-43f2-bb37-da4661949c13 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:40.694326 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ff446-e296-47c5-af16-5733c7841a18 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:40.714645 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe647d1c-93df-4eff-a878-9e14ea395d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:40.715323 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe647d1c-93df-4eff-a878-9e14ea395d11 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:40.716135 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe647d1c-93df-4eff-a878-9e14ea395d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:40.716135 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe647d1c-93df-4eff-a878-9e14ea395d11 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:40.716736 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe647d1c-93df-4eff-a878-9e14ea395d11 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:40.717048 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe647d1c-93df-4eff-a878-9e14ea395d11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:40.718454 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe647d1c-93df-4eff-a878-9e14ea395d11 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:40.725329 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 467/920] 10.222.0.60 () {56 vars in 1114 bytes} [Wed Jul 13 23:02:40 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:02:40.736906 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe647d1c-93df-4eff-a878-9e14ea395d11 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:40.744613 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea944a61-ab77-40ab-8be2-39f95dde3a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:40.745852 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea944a61-ab77-40ab-8be2-39f95dde3a09 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:40.745852 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea944a61-ab77-40ab-8be2-39f95dde3a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:40.745852 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea944a61-ab77-40ab-8be2-39f95dde3a09 None admin] PATH_INFO: `/v3/services` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:40.746591 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea944a61-ab77-40ab-8be2-39f95dde3a09 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:40.746902 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea944a61-ab77-40ab-8be2-39f95dde3a09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:40.748332 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea944a61-ab77-40ab-8be2-39f95dde3a09 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:40.754366 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 454/921] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Jul 13 23:02:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:02:40.761364 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea944a61-ab77-40ab-8be2-39f95dde3a09 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:40.882180 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:40.918562 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea944a61-ab77-40ab-8be2-39f95dde3a09 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:40.926916 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe647d1c-93df-4eff-a878-9e14ea395d11 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:40.965763 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fecb7ec-9186-4fec-8e6c-e688008589ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:40.966720 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fecb7ec-9186-4fec-8e6c-e688008589ae None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:40.967152 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fecb7ec-9186-4fec-8e6c-e688008589ae None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:40.967606 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fecb7ec-9186-4fec-8e6c-e688008589ae None admin] PATH_INFO: `/v3/endpoints` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:40.968511 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fecb7ec-9186-4fec-8e6c-e688008589ae None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:40.970080 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fecb7ec-9186-4fec-8e6c-e688008589ae None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:40.992842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9f5be-4567-4f4a-ba8d-df821670b889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:40.994003 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9f5be-4567-4f4a-ba8d-df821670b889 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:40.994548 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9f5be-4567-4f4a-ba8d-df821670b889 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:40.994939 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 468/922] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Jul 13 23:02:40 2022] POST /identity/v3/endpoints => generated 346 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:02:40.996106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9f5be-4567-4f4a-ba8d-df821670b889 None admin] PATH_INFO: `/v3/services` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:40.997074 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c9f5be-4567-4f4a-ba8d-df821670b889 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:40.997635 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c9f5be-4567-4f4a-ba8d-df821670b889 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:40.999332 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c9f5be-4567-4f4a-ba8d-df821670b889 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:41.006681 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 455/923] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Jul 13 23:02:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:41.013676 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fecb7ec-9186-4fec-8e6c-e688008589ae None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:41.033938 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:41.191280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fecb7ec-9186-4fec-8e6c-e688008589ae None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:41.262215 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09ddc4-ba11-4a8e-908f-156d59e888ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:41.263913 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09ddc4-ba11-4a8e-908f-156d59e888ce None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:41.264712 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09ddc4-ba11-4a8e-908f-156d59e888ce None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:41.265513 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09ddc4-ba11-4a8e-908f-156d59e888ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:41.267368 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe09ddc4-ba11-4a8e-908f-156d59e888ce None admin] RBAC: Adding query filter params (interface=public, service_id=ec2c6db652c345f9a1fc3f87f18533e8, region_id=RegionOne) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:41.268161 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe09ddc4-ba11-4a8e-908f-156d59e888ce None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:41.271014 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe09ddc4-ba11-4a8e-908f-156d59e888ce None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:41.282262 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 469/924] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Jul 13 23:02:41 2022] GET /identity/v3/endpoints?service_id=ec2c6db652c345f9a1fc3f87f18533e8&interface=public®ion_id=RegionOne => generated 194 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:02:41.288348 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9f5be-4567-4f4a-ba8d-df821670b889 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:41.488001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9f5be-4567-4f4a-ba8d-df821670b889 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:41.551074 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf65e40-b14a-4451-af3e-737a9e8ce553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:41.551827 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf65e40-b14a-4451-af3e-737a9e8ce553 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:41.552117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf65e40-b14a-4451-af3e-737a9e8ce553 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:41.552436 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf65e40-b14a-4451-af3e-737a9e8ce553 None admin] PATH_INFO: `/v3/services` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:41.553179 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf65e40-b14a-4451-af3e-737a9e8ce553 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:41.554645 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf65e40-b14a-4451-af3e-737a9e8ce553 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:41.562737 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 456/925] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Jul 13 23:02:41 2022] GET /identity/v3/services => generated 1497 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:02:42.343899 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddaf36c-a418-4877-8337-bc4834a4bef6 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:42.345440 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddaf36c-a418-4877-8337-bc4834a4bef6 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:42.346216 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddaf36c-a418-4877-8337-bc4834a4bef6 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:42.350196 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 470/926] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:42.355603 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca2ea81-7c3f-4753-a5a4-f6dc2ba6a3b4 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:42.356372 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca2ea81-7c3f-4753-a5a4-f6dc2ba6a3b4 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:42.356372 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca2ea81-7c3f-4753-a5a4-f6dc2ba6a3b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:42.379385 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-dca2ea81-7c3f-4753-a5a4-f6dc2ba6a3b4 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:42.381153 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-dca2ea81-7c3f-4753-a5a4-f6dc2ba6a3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:42.390570 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:42.461756 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 457/927] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 23:02:42.485495 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddaf36c-a418-4877-8337-bc4834a4bef6 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:42.578014 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:42.616672 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddaf36c-a418-4877-8337-bc4834a4bef6 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:42.657549 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a341f66-a589-4ff3-95d6-2f11eb85445d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:42.658740 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a341f66-a589-4ff3-95d6-2f11eb85445d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:42.659251 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a341f66-a589-4ff3-95d6-2f11eb85445d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:42.659774 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a341f66-a589-4ff3-95d6-2f11eb85445d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:42.660713 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a341f66-a589-4ff3-95d6-2f11eb85445d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:42.663018 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a341f66-a589-4ff3-95d6-2f11eb85445d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:42.676673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-3a341f66-a589-4ff3-95d6-2f11eb85445d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 23:02:42.676673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:42.676673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 23:02:42.676673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 23:02:42.676673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 23:02:42.676673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 23:02:42.676673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 23:02:42.676673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:42.676673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 23:02:42.676673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:42.676673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 23:02:42.676673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:42.676673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:42.676673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:42.676673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:42.676673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:42.677729 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:42.677729 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:42.677729 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:42.677729 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:42.677729 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:42.677729 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:42.677729 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:42.677729 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 23:02:42.677729 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 23:02:42.677729 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 23:02:42.677729 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 23:02:42.677729 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:42.677729 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:42.677729 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:42.677729 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:42.677729 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:42.678565 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:42.678565 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:42.678565 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:42.678565 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:42.678565 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:42.678565 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:42.678565 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:42.678565 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:42.678565 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:42.678565 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:42.678565 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:42.678565 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 23:02:42.678565 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:42.678565 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 23:02:42.678565 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:42.681648 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 471/928] 10.222.0.60 () {56 vars in 1132 bytes} [Wed Jul 13 23:02:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:42.689389 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca2ea81-7c3f-4753-a5a4-f6dc2ba6a3b4 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:42.807031 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca2ea81-7c3f-4753-a5a4-f6dc2ba6a3b4 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:42.846848 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf33125-0dc0-49b7-9192-d2855f0ccbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:42.848039 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf33125-0dc0-49b7-9192-d2855f0ccbc5 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:42.848546 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf33125-0dc0-49b7-9192-d2855f0ccbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:42.849026 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf33125-0dc0-49b7-9192-d2855f0ccbc5 None admin] PATH_INFO: `/v3/services` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:42.849989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf33125-0dc0-49b7-9192-d2855f0ccbc5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:42.850511 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf33125-0dc0-49b7-9192-d2855f0ccbc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:42.852151 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf33125-0dc0-49b7-9192-d2855f0ccbc5 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:42.858425 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 458/929] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Jul 13 23:02:42 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:42.864232 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a341f66-a589-4ff3-95d6-2f11eb85445d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:42.983987 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a341f66-a589-4ff3-95d6-2f11eb85445d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:43.024608 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e3ae16-ced3-4a84-893b-d52e970f8ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:43.025685 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3ae16-ced3-4a84-893b-d52e970f8ceb None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:43.026187 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3ae16-ced3-4a84-893b-d52e970f8ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:43.026673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3ae16-ced3-4a84-893b-d52e970f8ceb None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:43.027685 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e3ae16-ced3-4a84-893b-d52e970f8ceb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:43.028240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e3ae16-ced3-4a84-893b-d52e970f8ceb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:43.029894 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e3ae16-ced3-4a84-893b-d52e970f8ceb None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:43.032845 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:43.037449 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 472/930] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Jul 13 23:02:42 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:43.044653 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf33125-0dc0-49b7-9192-d2855f0ccbc5 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:43.057494 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:43.163473 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf33125-0dc0-49b7-9192-d2855f0ccbc5 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:43.206133 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd4c6d8-cf6d-451e-ae22-235b7f039c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:43.207222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4c6d8-cf6d-451e-ae22-235b7f039c92 None admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:43.207760 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4c6d8-cf6d-451e-ae22-235b7f039c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:43.208244 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4c6d8-cf6d-451e-ae22-235b7f039c92 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:43.209257 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd4c6d8-cf6d-451e-ae22-235b7f039c92 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:43.210896 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd4c6d8-cf6d-451e-ae22-235b7f039c92 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:43.236309 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 459/931] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Jul 13 23:02:43 2022] POST /identity/v3/endpoints => generated 347 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:02:43.249136 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aafdae-f3e7-4ddd-bf54-70e8b756dbbe None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:43.249641 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aafdae-f3e7-4ddd-bf54-70e8b756dbbe None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:43.250125 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aafdae-f3e7-4ddd-bf54-70e8b756dbbe None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:43.259732 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 473/932] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:43.263408 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3267d-7882-4969-a1dd-601e94e1b9f6 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:43.263952 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3267d-7882-4969-a1dd-601e94e1b9f6 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:43.264423 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3267d-7882-4969-a1dd-601e94e1b9f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:43.293744 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-f8c3267d-7882-4969-a1dd-601e94e1b9f6 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:43.295932 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-f8c3267d-7882-4969-a1dd-601e94e1b9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:43.387970 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:43.406393 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 460/933] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:02:43.447199 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aafdae-f3e7-4ddd-bf54-70e8b756dbbe None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:43.563062 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aafdae-f3e7-4ddd-bf54-70e8b756dbbe None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:43.601424 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd9c29-1862-4a11-9c47-9a51277aeadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:43.602654 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd9c29-1862-4a11-9c47-9a51277aeadd None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:43.603212 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd9c29-1862-4a11-9c47-9a51277aeadd None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:43.603726 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd9c29-1862-4a11-9c47-9a51277aeadd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:43.617033 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd9c29-1862-4a11-9c47-9a51277aeadd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:02:43.625702 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-91fd9c29-1862-4a11-9c47-9a51277aeadd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 23:02:43.625702 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:43.625702 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:43.625702 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:43.625702 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:43.625702 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:43.625702 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:43.625702 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:43.625702 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:43.625702 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:43.625702 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:43.625702 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:43.625702 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:02:43.625702 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:02:43.625702 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:02:43.625702 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:02:43.626703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:02:43.626703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:02:43.626703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:02:43.626703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:02:43.626703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:43.626703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:43.626703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:43.626703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:43.626703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:43.626703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:43.626703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:43.626703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:43.626703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:43.626703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:43.626703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:43.626703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:43.627567 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:43.627567 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:43.627567 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:43.627567 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:43.627567 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:02:43.627567 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:02:43.627567 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:02:43.627567 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:02:43.627567 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:02:43.627567 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:02:43.627567 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 23:02:43.627567 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:43.630464 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 474/934] 10.222.0.60 () {56 vars in 1120 bytes} [Wed Jul 13 23:02:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:43.640660 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c3267d-7882-4969-a1dd-601e94e1b9f6 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:43.773213 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c3267d-7882-4969-a1dd-601e94e1b9f6 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:43.811854 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df3eadf-8947-4bde-a51a-786e667d437e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:43.812546 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df3eadf-8947-4bde-a51a-786e667d437e None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:43.812828 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df3eadf-8947-4bde-a51a-786e667d437e None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:43.813162 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df3eadf-8947-4bde-a51a-786e667d437e None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:43.813877 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df3eadf-8947-4bde-a51a-786e667d437e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:43.814183 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df3eadf-8947-4bde-a51a-786e667d437e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:43.815614 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df3eadf-8947-4bde-a51a-786e667d437e None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:43.826014 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 461/935] 10.222.0.60 () {56 vars in 1115 bytes} [Wed Jul 13 23:02:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:43.835244 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd9c29-1862-4a11-9c47-9a51277aeadd None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:43.925762 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:43.930329 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed2781-05b0-4f41-be8f-21f8d0164d3e None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:43.930737 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed2781-05b0-4f41-be8f-21f8d0164d3e None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:43.931030 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed2781-05b0-4f41-be8f-21f8d0164d3e None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:43.940288 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 462/936] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:43 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:02:43.943101 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19b911-6c52-4fa5-8d7c-523ec032422f None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:43.943668 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19b911-6c52-4fa5-8d7c-523ec032422f None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:43.944158 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19b911-6c52-4fa5-8d7c-523ec032422f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:43.962030 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd9c29-1862-4a11-9c47-9a51277aeadd None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:43.973108 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-af19b911-6c52-4fa5-8d7c-523ec032422f None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:43.975778 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-af19b911-6c52-4fa5-8d7c-523ec032422f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:44.001066 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3626647-e8a9-4bf8-a356-e5d671b58b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:44.001892 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3626647-e8a9-4bf8-a356-e5d671b58b8b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:44.002290 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3626647-e8a9-4bf8-a356-e5d671b58b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:44.002577 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3626647-e8a9-4bf8-a356-e5d671b58b8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:44.014755 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3626647-e8a9-4bf8-a356-e5d671b58b8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:44.016280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3626647-e8a9-4bf8-a356-e5d671b58b8b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:44.037811 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-a3626647-e8a9-4bf8-a356-e5d671b58b8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:44.037811 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:44.037811 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:44.037811 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:44.037811 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:44.037811 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:44.037811 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:44.037811 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:44.037811 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:44.037811 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:44.037811 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:44.037811 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:44.037811 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:44.037811 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:44.037811 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:44.038751 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:44.038751 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:44.038751 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:44.038751 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:44.038751 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:44.038751 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:44.038751 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:44.038751 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:44.038751 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:44.038751 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:44.038751 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:44.038751 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:44.038751 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:44.038751 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:44.038751 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:44.038751 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:44.038751 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:44.040132 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:44.040132 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:44.040132 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:44.040132 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:44.044074 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 475/937] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:44.045254 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3626647-e8a9-4bf8-a356-e5d671b58b8b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:44.063849 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:44.112247 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 463/938] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 23:02:44.144026 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19b911-6c52-4fa5-8d7c-523ec032422f None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:44.171100 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3626647-e8a9-4bf8-a356-e5d671b58b8b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:44.186947 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:44.220631 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accd1ac9-aa8d-4c86-8e3e-65d2149f232b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:44.221450 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd1ac9-aa8d-4c86-8e3e-65d2149f232b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:44.221828 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd1ac9-aa8d-4c86-8e3e-65d2149f232b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:44.222455 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd1ac9-aa8d-4c86-8e3e-65d2149f232b None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:44.223859 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accd1ac9-aa8d-4c86-8e3e-65d2149f232b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:44.224951 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accd1ac9-aa8d-4c86-8e3e-65d2149f232b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:44.226762 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accd1ac9-aa8d-4c86-8e3e-65d2149f232b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:44.247586 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 476/939] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:02:44.251608 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accd1ac9-aa8d-4c86-8e3e-65d2149f232b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:44.292577 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19b911-6c52-4fa5-8d7c-523ec032422f None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:44.337849 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a32e2f-82d2-4fbe-a489-dd23bf6bb70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:44.339833 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a32e2f-82d2-4fbe-a489-dd23bf6bb70f None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:44.340349 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a32e2f-82d2-4fbe-a489-dd23bf6bb70f None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:44.341105 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a32e2f-82d2-4fbe-a489-dd23bf6bb70f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:44.342512 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a32e2f-82d2-4fbe-a489-dd23bf6bb70f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:44.344803 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a32e2f-82d2-4fbe-a489-dd23bf6bb70f None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:44.361757 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-93a32e2f-82d2-4fbe-a489-dd23bf6bb70f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 23:02:44.361757 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:44.361757 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 23:02:44.361757 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 23:02:44.361757 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 23:02:44.361757 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 23:02:44.361757 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 23:02:44.361757 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:44.361757 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 23:02:44.361757 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:44.361757 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 23:02:44.361757 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:44.361757 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:44.361757 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:44.361757 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:44.361757 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:44.361757 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:44.362787 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:44.362787 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:44.362787 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:44.362787 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:44.362787 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:44.362787 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:44.362787 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 23:02:44.362787 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 23:02:44.362787 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 23:02:44.362787 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 23:02:44.362787 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:44.362787 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:44.362787 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:44.362787 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:44.362787 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:44.362787 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:44.363645 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:44.363645 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:44.363645 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:44.363645 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:44.363645 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:44.363645 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:44.363645 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:44.363645 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:44.363645 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:44.363645 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:44.363645 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 23:02:44.363645 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:44.363645 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 23:02:44.363645 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:44.367053 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 464/940] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Jul 13 23:02:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:44.375997 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a32e2f-82d2-4fbe-a489-dd23bf6bb70f None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:44.400852 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accd1ac9-aa8d-4c86-8e3e-65d2149f232b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:44.432473 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:44.462552 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839975a6-e399-4221-8c4c-83e54c779a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:44.465333 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839975a6-e399-4221-8c4c-83e54c779a35 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:44.465913 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839975a6-e399-4221-8c4c-83e54c779a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:44.466390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839975a6-e399-4221-8c4c-83e54c779a35 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:44.479452 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839975a6-e399-4221-8c4c-83e54c779a35 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:44.481018 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839975a6-e399-4221-8c4c-83e54c779a35 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:44.498237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-839975a6-e399-4221-8c4c-83e54c779a35 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 23:02:44.498237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:44.498237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:44.498237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:44.498237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:44.498237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:44.498237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:44.498237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:44.498237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:44.498237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:44.498237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:44.498237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:44.498237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 23:02:44.498237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 23:02:44.498237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 23:02:44.498237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 23:02:44.499241 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:44.499241 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:44.499241 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 23:02:44.499241 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:44.499241 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 23:02:44.499241 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:44.499241 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:44.499241 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:44.499241 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:44.499241 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:44.499241 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:44.499241 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:44.499241 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:44.499241 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:44.499241 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:44.499241 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:44.499241 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:44.499241 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:44.500828 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:44.500828 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:44.500828 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 23:02:44.500828 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 23:02:44.500828 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 23:02:44.500828 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 23:02:44.500828 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 23:02:44.500828 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 23:02:44.500828 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 23:02:44.500828 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:44.503815 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 477/941] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Jul 13 23:02:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:02:44.506273 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839975a6-e399-4221-8c4c-83e54c779a35 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:44.509596 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a32e2f-82d2-4fbe-a489-dd23bf6bb70f None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:44.542629 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:44.564549 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f15cd-0bba-4dd4-a1b5-5ab1d8de0373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:44.566033 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f15cd-0bba-4dd4-a1b5-5ab1d8de0373 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:44.566534 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f15cd-0bba-4dd4-a1b5-5ab1d8de0373 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:44.567011 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f15cd-0bba-4dd4-a1b5-5ab1d8de0373 None admin] PATH_INFO: `/v3/services` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:44.569175 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51f15cd-0bba-4dd4-a1b5-5ab1d8de0373 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:44.569759 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51f15cd-0bba-4dd4-a1b5-5ab1d8de0373 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:44.572252 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51f15cd-0bba-4dd4-a1b5-5ab1d8de0373 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:44.585518 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 465/942] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Jul 13 23:02:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:02:44.587624 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f15cd-0bba-4dd4-a1b5-5ab1d8de0373 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:44.628720 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839975a6-e399-4221-8c4c-83e54c779a35 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:44.673227 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78415a-e179-431e-8420-3802b094e65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:44.673998 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78415a-e179-431e-8420-3802b094e65f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:44.674296 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78415a-e179-431e-8420-3802b094e65f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:44.674564 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78415a-e179-431e-8420-3802b094e65f None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:44.675412 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d78415a-e179-431e-8420-3802b094e65f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:44.675856 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d78415a-e179-431e-8420-3802b094e65f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:44.677860 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d78415a-e179-431e-8420-3802b094e65f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:44.705521 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 478/943] 10.222.0.60 () {56 vars in 1149 bytes} [Wed Jul 13 23:02:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:02:44.705521 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb8230-b22a-4131-8c57-0a9f5a6f73eb None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:44.705521 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb8230-b22a-4131-8c57-0a9f5a6f73eb None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:44.705521 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb8230-b22a-4131-8c57-0a9f5a6f73eb None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:44.705521 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 479/944] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:44.709163 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb8230-b22a-4131-8c57-0a9f5a6f73eb None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:44.727827 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f15cd-0bba-4dd4-a1b5-5ab1d8de0373 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:44.738787 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:44.783656 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d34bfb-3299-47ef-9bed-16a10a50758f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:44.784847 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d34bfb-3299-47ef-9bed-16a10a50758f None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:44.785405 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d34bfb-3299-47ef-9bed-16a10a50758f None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:44.785926 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d34bfb-3299-47ef-9bed-16a10a50758f None admin] PATH_INFO: `/v3/services` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:44.787325 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d34bfb-3299-47ef-9bed-16a10a50758f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:44.787938 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d34bfb-3299-47ef-9bed-16a10a50758f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:44.790218 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d34bfb-3299-47ef-9bed-16a10a50758f None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:44.799079 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 466/945] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Jul 13 23:02:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:02:44.806833 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d4049-027e-4746-a9b1-78b4d9498f94 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:44.807371 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d4049-027e-4746-a9b1-78b4d9498f94 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:44.807820 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d4049-027e-4746-a9b1-78b4d9498f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:44.840718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-e87d4049-027e-4746-a9b1-78b4d9498f94 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:44.842935 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-e87d4049-027e-4746-a9b1-78b4d9498f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:44.845276 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:44.859088 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb8230-b22a-4131-8c57-0a9f5a6f73eb None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:44.908742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38abd56-2c54-41c2-a54c-b0a9c231f882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:44.909729 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38abd56-2c54-41c2-a54c-b0a9c231f882 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:44.910183 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38abd56-2c54-41c2-a54c-b0a9c231f882 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:44.910592 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38abd56-2c54-41c2-a54c-b0a9c231f882 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:44.924251 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38abd56-2c54-41c2-a54c-b0a9c231f882 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:44.925981 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38abd56-2c54-41c2-a54c-b0a9c231f882 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:44.948276 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-f38abd56-2c54-41c2-a54c-b0a9c231f882 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:44.948276 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:44.948276 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:44.948276 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:44.948276 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:44.948276 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:44.948276 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:44.948276 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:44.948276 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:44.948276 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:44.948276 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:44.948276 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:44.948276 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:44.948276 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:44.948276 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:44.949980 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:44.949980 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:44.949980 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:44.949980 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:44.949980 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:44.949980 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:44.949980 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:44.949980 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:44.949980 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:44.949980 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:44.949980 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:44.949980 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:44.949980 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:44.949980 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:44.949980 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:44.949980 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:44.949980 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:44.951740 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:44.951740 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:44.951740 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:44.951740 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:44.958133 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 480/946] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:44.958133 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38abd56-2c54-41c2-a54c-b0a9c231f882 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:44.991566 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 467/947] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:02:44.991566 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d4049-027e-4746-a9b1-78b4d9498f94 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:45.131191 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38abd56-2c54-41c2-a54c-b0a9c231f882 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:45.141966 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d4049-027e-4746-a9b1-78b4d9498f94 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:45.145169 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:45.182107 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0538cbe7-9dcf-4379-a024-8d3f01f1f300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:45.182873 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0538cbe7-9dcf-4379-a024-8d3f01f1f300 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:45.183273 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0538cbe7-9dcf-4379-a024-8d3f01f1f300 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:45.183718 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0538cbe7-9dcf-4379-a024-8d3f01f1f300 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:45.184811 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0538cbe7-9dcf-4379-a024-8d3f01f1f300 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:45.186600 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0538cbe7-9dcf-4379-a024-8d3f01f1f300 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:45.205059 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:45.206959 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393f276-6cca-40cf-a630-40472d9d0515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:45.209047 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393f276-6cca-40cf-a630-40472d9d0515 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:45.209669 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393f276-6cca-40cf-a630-40472d9d0515 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:45.210185 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393f276-6cca-40cf-a630-40472d9d0515 None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:45.211749 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393f276-6cca-40cf-a630-40472d9d0515 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:45.212309 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393f276-6cca-40cf-a630-40472d9d0515 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:45.214110 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 481/948] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Jul 13 23:02:44 2022] POST /identity/v3/endpoints => generated 349 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:02:45.214916 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393f276-6cca-40cf-a630-40472d9d0515 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:45.224202 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0538cbe7-9dcf-4379-a024-8d3f01f1f300 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:45.251752 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 468/949] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:45.253462 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393f276-6cca-40cf-a630-40472d9d0515 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:45.454685 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393f276-6cca-40cf-a630-40472d9d0515 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:45.469218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0538cbe7-9dcf-4379-a024-8d3f01f1f300 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:45.524729 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b8e761-2149-4aa1-adc3-74f356b2aeb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:45.526117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8e761-2149-4aa1-adc3-74f356b2aeb6 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:45.526540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8e761-2149-4aa1-adc3-74f356b2aeb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:45.526937 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8e761-2149-4aa1-adc3-74f356b2aeb6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:45.538462 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b8e761-2149-4aa1-adc3-74f356b2aeb6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:45.540629 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b8e761-2149-4aa1-adc3-74f356b2aeb6 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:45.548356 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc05a93b-2938-45e5-b962-3decd3e0f1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:45.549892 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05a93b-2938-45e5-b962-3decd3e0f1d0 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:45.550639 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05a93b-2938-45e5-b962-3decd3e0f1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:45.551378 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05a93b-2938-45e5-b962-3decd3e0f1d0 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:45.553545 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc05a93b-2938-45e5-b962-3decd3e0f1d0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:45.557007 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc05a93b-2938-45e5-b962-3decd3e0f1d0 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:45.559959 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: WARNING keystone.server.flask.common [None req-cc05a93b-2938-45e5-b962-3decd3e0f1d0 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 13 23:02:45.572317 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-03b8e761-2149-4aa1-adc3-74f356b2aeb6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:45.572317 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:45.572317 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:45.572317 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:45.572317 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:45.572317 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:45.572317 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:45.572317 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:45.572317 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:45.572317 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:45.572317 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:45.572317 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:45.572317 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 23:02:45.572317 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 23:02:45.572317 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 23:02:45.573968 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 23:02:45.573968 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:45.573968 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:45.573968 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:45.573968 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:45.573968 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:45.573968 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:45.573968 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:45.573968 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:45.573968 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:45.573968 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:45.573968 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:45.573968 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:45.573968 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:45.573968 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:45.573968 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:45.573968 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:45.575591 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 23:02:45.575591 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 23:02:45.575591 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 23:02:45.575591 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 23:02:45.575591 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 23:02:45.575591 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 23:02:45.575591 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:45.575591 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:45.575591 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 469/950] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:02:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:45.584571 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b8e761-2149-4aa1-adc3-74f356b2aeb6 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:45.605841 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.sql.core [None req-cc05a93b-2938-45e5-b962-3decd3e0f1d0 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 13 23:02:45.605841 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 13 23:02:45.605841 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [parameters: {'id': '29cb3313e19e4561998eaa80f94c7fb6', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 13 23:02:45.605841 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80284) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 13 23:02:45.628510 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-cc05a93b-2938-45e5-b962-3decd3e0f1d0 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 29cb3313e19e4561998eaa80f94c7fb6.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 29cb3313e19e4561998eaa80f94c7fb6. Jul 13 23:02:45.628510 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:45.628510 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 13 23:02:45.628510 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 13 23:02:45.628510 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 13 23:02:45.628510 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 13 23:02:45.628510 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 13 23:02:45.628510 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application result = self._query(query) Jul 13 23:02:45.628510 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 13 23:02:45.628510 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application conn.query(q) Jul 13 23:02:45.628510 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 13 23:02:45.628510 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 13 23:02:45.628510 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 13 23:02:45.630067 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application result.read() Jul 13 23:02:45.630067 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 13 23:02:45.630067 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 13 23:02:45.630067 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 13 23:02:45.630067 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 13 23:02:45.630067 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 13 23:02:45.630067 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 13 23:02:45.630067 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 13 23:02:45.630067 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 13 23:02:45.630067 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 13 23:02:45.630067 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:45.630067 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 13 23:02:45.630067 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:45.630067 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:45.630067 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 13 23:02:45.630067 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 13 23:02:45.630067 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 13 23:02:45.630067 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 13 23:02:45.632471 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 13 23:02:45.632471 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application next(self.gen) Jul 13 23:02:45.632471 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 13 23:02:45.632471 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application yield resource Jul 13 23:02:45.632471 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 13 23:02:45.632471 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application next(self.gen) Jul 13 23:02:45.632471 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 13 23:02:45.632471 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self.session.rollback() Jul 13 23:02:45.632471 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 13 23:02:45.632471 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self.force_reraise() Jul 13 23:02:45.632471 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 13 23:02:45.632471 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise self.value Jul 13 23:02:45.632471 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 13 23:02:45.632471 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 13 23:02:45.632471 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 13 23:02:45.632471 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application session.commit() Jul 13 23:02:45.632471 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 13 23:02:45.634159 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 13 23:02:45.634159 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 13 23:02:45.634159 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._prepare_impl() Jul 13 23:02:45.634159 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 13 23:02:45.634159 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self.session.flush() Jul 13 23:02:45.634159 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 13 23:02:45.634159 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._flush(objects) Jul 13 23:02:45.634159 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 13 23:02:45.634159 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 13 23:02:45.634159 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 13 23:02:45.634159 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application compat.raise_( Jul 13 23:02:45.634159 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 13 23:02:45.634159 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception Jul 13 23:02:45.634159 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 13 23:02:45.634159 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application flush_context.execute() Jul 13 23:02:45.634159 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 13 23:02:45.634159 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rec.execute(self) Jul 13 23:02:45.638088 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 13 23:02:45.638088 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 13 23:02:45.638088 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 13 23:02:45.638088 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 13 23:02:45.638088 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 13 23:02:45.638088 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 13 23:02:45.638088 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 13 23:02:45.638088 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 13 23:02:45.638088 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 13 23:02:45.638088 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 13 23:02:45.638088 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 13 23:02:45.638088 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 13 23:02:45.638088 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 13 23:02:45.638088 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 13 23:02:45.638088 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 13 23:02:45.641339 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 13 23:02:45.641339 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 13 23:02:45.641339 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception Jul 13 23:02:45.641339 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 13 23:02:45.641339 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 13 23:02:45.641339 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 13 23:02:45.641339 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 13 23:02:45.641339 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 13 23:02:45.641339 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application result = self._query(query) Jul 13 23:02:45.641339 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 13 23:02:45.641339 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application conn.query(q) Jul 13 23:02:45.641339 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 13 23:02:45.641339 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 13 23:02:45.641339 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 13 23:02:45.641339 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application result.read() Jul 13 23:02:45.641339 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 13 23:02:45.641339 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 13 23:02:45.645153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 13 23:02:45.645153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 13 23:02:45.645153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 13 23:02:45.645153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 13 23:02:45.645153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 13 23:02:45.645153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 13 23:02:45.645153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 13 23:02:45.645153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 13 23:02:45.645153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [parameters: {'id': '29cb3313e19e4561998eaa80f94c7fb6', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 13 23:02:45.645153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 13 23:02:45.645153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:45.645153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 23:02:45.645153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:45.645153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:45.645153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 13 23:02:45.647654 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 13 23:02:45.647654 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 13 23:02:45.647654 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 13 23:02:45.647654 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 13 23:02:45.647654 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:45.647654 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 23:02:45.647654 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:45.647654 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:45.647654 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:45.647654 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:45.647654 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:45.647654 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:45.647654 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:45.647654 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:45.647654 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:45.647654 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:45.647654 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:45.651112 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:45.651112 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 13 23:02:45.651112 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 13 23:02:45.651112 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:45.651112 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:45.651112 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 13 23:02:45.651112 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 13 23:02:45.651112 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 29cb3313e19e4561998eaa80f94c7fb6. Jul 13 23:02:45.651112 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:45.651112 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 482/951] 10.222.0.60 () {60 vars in 1140 bytes} [Wed Jul 13 23:02:45 2022] POST /identity/v3/projects => generated 262 bytes in 420 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:45.651112 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc05a93b-2938-45e5-b962-3decd3e0f1d0 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:45.655195 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:45.745865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:45.835954 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b8e761-2149-4aa1-adc3-74f356b2aeb6 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:45.855560 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc05a93b-2938-45e5-b962-3decd3e0f1d0 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:45.903674 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64cee94-5975-4755-8c84-7ea1efa5934f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:45.905395 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64cee94-5975-4755-8c84-7ea1efa5934f None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:45.906618 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64cee94-5975-4755-8c84-7ea1efa5934f None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:45.907314 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64cee94-5975-4755-8c84-7ea1efa5934f None admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:45.909318 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64cee94-5975-4755-8c84-7ea1efa5934f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:45.910087 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64cee94-5975-4755-8c84-7ea1efa5934f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:45.914753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64cee94-5975-4755-8c84-7ea1efa5934f None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:45.926972 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e3f34-f8e4-4a02-9d55-fa1435b4a493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:45.929095 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e3f34-f8e4-4a02-9d55-fa1435b4a493 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:45.929681 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e3f34-f8e4-4a02-9d55-fa1435b4a493 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:45.930943 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e3f34-f8e4-4a02-9d55-fa1435b4a493 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:45.940150 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 470/952] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jul 13 23:02:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:45.943386 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953e3f34-f8e4-4a02-9d55-fa1435b4a493 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:45.945578 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953e3f34-f8e4-4a02-9d55-fa1435b4a493 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:45.955585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64cee94-5975-4755-8c84-7ea1efa5934f None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:45.984701 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-953e3f34-f8e4-4a02-9d55-fa1435b4a493 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:45.984701 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:45.984701 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:45.984701 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:45.984701 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:45.984701 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:45.984701 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:45.984701 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:45.984701 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:45.984701 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:45.984701 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:45.984701 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:45.984701 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 23:02:45.984701 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 23:02:45.984701 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 23:02:45.986908 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 23:02:45.986908 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:45.986908 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:45.986908 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:45.986908 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:45.986908 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:45.986908 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:45.986908 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:45.986908 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:45.986908 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:45.986908 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:45.986908 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:45.986908 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:45.986908 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:45.986908 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:45.986908 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:45.986908 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:45.989013 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 23:02:45.989013 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 23:02:45.989013 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 23:02:45.989013 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 23:02:45.989013 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 23:02:45.989013 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 23:02:45.989013 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:45.989013 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:45.995685 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 483/953] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:02:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:45.998905 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e3f34-f8e4-4a02-9d55-fa1435b4a493 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:46.144036 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:46.178051 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64cee94-5975-4755-8c84-7ea1efa5934f None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:46.207015 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e3f34-f8e4-4a02-9d55-fa1435b4a493 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:46.231801 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:46.263427 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f247ad2-0bcd-4988-9c86-85302c9e597c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:46.265918 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f247ad2-0bcd-4988-9c86-85302c9e597c None admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:46.267117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f247ad2-0bcd-4988-9c86-85302c9e597c None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:46.268609 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f247ad2-0bcd-4988-9c86-85302c9e597c None admin] PATH_INFO: `/v3/projects/ab0a5581e4e444c293815e1304e20c05/users/a32702a67a7b49ffa153fc59251fd169/roles/f7411bbcd41a4f959f76b130485bfb89` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:46.275338 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f247ad2-0bcd-4988-9c86-85302c9e597c None admin] RBAC: Authorizing `identity:create_grant(project_id=ab0a5581e4e444c293815e1304e20c05, user_id=a32702a67a7b49ffa153fc59251fd169, role_id=f7411bbcd41a4f959f76b130485bfb89)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:46.279370 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f247ad2-0bcd-4988-9c86-85302c9e597c None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:46.281925 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c7718-0c0c-48a7-b9c7-57583e9d4734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:46.282999 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c7718-0c0c-48a7-b9c7-57583e9d4734 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:46.283505 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c7718-0c0c-48a7-b9c7-57583e9d4734 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:46.284053 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c7718-0c0c-48a7-b9c7-57583e9d4734 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:46.285378 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463c7718-0c0c-48a7-b9c7-57583e9d4734 None admin] RBAC: Adding query filter params (name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:46.285986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463c7718-0c0c-48a7-b9c7-57583e9d4734 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:46.289806 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463c7718-0c0c-48a7-b9c7-57583e9d4734 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:46.313147 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 471/954] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jul 13 23:02:45 2022] PUT /identity/v3/projects/ab0a5581e4e444c293815e1304e20c05/users/a32702a67a7b49ffa153fc59251fd169/roles/f7411bbcd41a4f959f76b130485bfb89 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:02:46.337207 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 484/955] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Jul 13 23:02:45 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:02:46.410941 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf1e1d-663d-4a5b-bbde-8a8e90e45139 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:46.411357 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf1e1d-663d-4a5b-bbde-8a8e90e45139 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:46.412313 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf1e1d-663d-4a5b-bbde-8a8e90e45139 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:46.414087 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 472/956] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:46.422998 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd15bc9-0927-4565-afab-8a75c7a1e100 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:46.423892 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd15bc9-0927-4565-afab-8a75c7a1e100 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:46.424531 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd15bc9-0927-4565-afab-8a75c7a1e100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:46.453500 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-7dd15bc9-0927-4565-afab-8a75c7a1e100 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:46.456466 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7dd15bc9-0927-4565-afab-8a75c7a1e100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:46.600753 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 485/957] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:02:46.638878 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf1e1d-663d-4a5b-bbde-8a8e90e45139 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:46.844363 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf1e1d-663d-4a5b-bbde-8a8e90e45139 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:46.910398 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7bb7e-ed14-4c86-97be-94242965b048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:46.911820 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7bb7e-ed14-4c86-97be-94242965b048 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:46.912309 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7bb7e-ed14-4c86-97be-94242965b048 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:46.912792 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7bb7e-ed14-4c86-97be-94242965b048 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:46.915315 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea7bb7e-ed14-4c86-97be-94242965b048 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:46.917884 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea7bb7e-ed14-4c86-97be-94242965b048 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:46.941441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-fea7bb7e-ed14-4c86-97be-94242965b048 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 23:02:46.941441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:46.941441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 23:02:46.941441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 23:02:46.941441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 23:02:46.941441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 23:02:46.941441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 23:02:46.941441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:46.941441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 23:02:46.941441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:46.941441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 23:02:46.941441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:46.941441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:46.941441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:46.941441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:46.941441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:46.941441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:46.944944 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:46.944944 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:46.944944 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:46.944944 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:46.944944 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:46.944944 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:46.944944 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 23:02:46.944944 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 23:02:46.944944 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 23:02:46.944944 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 23:02:46.944944 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:46.944944 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:46.944944 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:46.944944 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:46.944944 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:46.944944 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:46.948501 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:46.948501 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:46.948501 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:46.948501 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:46.948501 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:46.948501 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:46.948501 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:46.948501 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:46.948501 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:46.948501 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:46.948501 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 23:02:46.948501 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:46.948501 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 23:02:46.948501 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:46.948501 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 473/958] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:02:46 2022] GET /identity/v3/services/compute => generated 88 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:46.957296 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd15bc9-0927-4565-afab-8a75c7a1e100 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:47.146466 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:47.151690 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd15bc9-0927-4565-afab-8a75c7a1e100 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:47.218033 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31bf838-7bbb-40a5-9aee-1cddcd16f941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:47.219026 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31bf838-7bbb-40a5-9aee-1cddcd16f941 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:47.219521 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31bf838-7bbb-40a5-9aee-1cddcd16f941 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:47.220045 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31bf838-7bbb-40a5-9aee-1cddcd16f941 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:47.221334 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31bf838-7bbb-40a5-9aee-1cddcd16f941 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:47.221869 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31bf838-7bbb-40a5-9aee-1cddcd16f941 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:47.225029 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31bf838-7bbb-40a5-9aee-1cddcd16f941 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:47.235137 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 486/959] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Jul 13 23:02:46 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:47.244675 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7bb7e-ed14-4c86-97be-94242965b048 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:47.268363 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:47.455955 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7bb7e-ed14-4c86-97be-94242965b048 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:47.519720 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a8898-01c3-4967-a27b-5075902a9297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:47.520811 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a8898-01c3-4967-a27b-5075902a9297 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:47.521315 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a8898-01c3-4967-a27b-5075902a9297 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:47.521813 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a8898-01c3-4967-a27b-5075902a9297 None admin] PATH_INFO: `/v3/services` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:47.523231 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a8898-01c3-4967-a27b-5075902a9297 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:47.523817 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a8898-01c3-4967-a27b-5075902a9297 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:47.526450 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a8898-01c3-4967-a27b-5075902a9297 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:47.536882 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 474/960] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Jul 13 23:02:47 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:48.754152 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d3f34-8f8a-4171-b6f4-a1a8c10b3b1c None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:48.754957 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d3f34-8f8a-4171-b6f4-a1a8c10b3b1c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:48.755137 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d3f34-8f8a-4171-b6f4-a1a8c10b3b1c None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:48.758149 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 487/961] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:48.766870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d688a25-3aac-4469-8090-ca01d2fd4605 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:48.767655 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d688a25-3aac-4469-8090-ca01d2fd4605 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:48.767950 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d688a25-3aac-4469-8090-ca01d2fd4605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:48.795962 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-0d688a25-3aac-4469-8090-ca01d2fd4605 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:48.798007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-0d688a25-3aac-4469-8090-ca01d2fd4605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:48.886564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:48.891998 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 475/962] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 23:02:48.914105 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d3f34-8f8a-4171-b6f4-a1a8c10b3b1c None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:49.094021 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d3f34-8f8a-4171-b6f4-a1a8c10b3b1c None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:49.156977 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984121d8-afa7-4e12-8769-e022acc4c07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:49.158342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984121d8-afa7-4e12-8769-e022acc4c07c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:49.158836 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984121d8-afa7-4e12-8769-e022acc4c07c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:49.159333 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984121d8-afa7-4e12-8769-e022acc4c07c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:49.160614 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984121d8-afa7-4e12-8769-e022acc4c07c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:49.163805 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984121d8-afa7-4e12-8769-e022acc4c07c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:49.186240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-984121d8-afa7-4e12-8769-e022acc4c07c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 23:02:49.186240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:49.186240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 23:02:49.186240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 23:02:49.186240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 23:02:49.186240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 23:02:49.186240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 23:02:49.186240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:49.186240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 23:02:49.186240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:49.186240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 23:02:49.186240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:49.186240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:49.186240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:49.186240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:49.186240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:49.186240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:49.188278 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:49.188278 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:49.188278 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:49.188278 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:49.188278 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:49.188278 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:49.188278 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 23:02:49.188278 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 23:02:49.188278 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 23:02:49.188278 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 23:02:49.188278 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:49.188278 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:49.188278 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:49.188278 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:49.188278 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:49.188278 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:49.189459 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:49.189459 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:49.189459 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:49.189459 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:49.189459 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:49.189459 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:49.189459 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:49.189459 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:49.189459 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:49.189459 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:49.189459 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 23:02:49.189459 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:49.189459 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 23:02:49.189459 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:49.189459 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 488/963] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Jul 13 23:02:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:49.194786 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d688a25-3aac-4469-8090-ca01d2fd4605 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:49.352996 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d688a25-3aac-4469-8090-ca01d2fd4605 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:49.392750 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d006a15f-8be0-4b16-b1fa-28e6f6e0b97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:49.393424 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d006a15f-8be0-4b16-b1fa-28e6f6e0b97a None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:49.393706 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d006a15f-8be0-4b16-b1fa-28e6f6e0b97a None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:49.393981 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d006a15f-8be0-4b16-b1fa-28e6f6e0b97a None admin] PATH_INFO: `/v3/services` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:49.394707 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d006a15f-8be0-4b16-b1fa-28e6f6e0b97a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:49.395015 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d006a15f-8be0-4b16-b1fa-28e6f6e0b97a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:49.396991 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d006a15f-8be0-4b16-b1fa-28e6f6e0b97a None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:49.402877 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 476/964] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Jul 13 23:02:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:49.410233 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984121d8-afa7-4e12-8769-e022acc4c07c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:49.479928 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:49.528881 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347dc5f8-5bc4-43ed-b521-dabd63e38bdc None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:49.529279 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347dc5f8-5bc4-43ed-b521-dabd63e38bdc None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:49.529644 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347dc5f8-5bc4-43ed-b521-dabd63e38bdc None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:49.532471 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 477/965] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:49.536946 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba935699-d93a-42c8-8caa-74384508335c None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:49.537296 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba935699-d93a-42c8-8caa-74384508335c None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:49.537645 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba935699-d93a-42c8-8caa-74384508335c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:49.555088 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-ba935699-d93a-42c8-8caa-74384508335c None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:49.556611 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-ba935699-d93a-42c8-8caa-74384508335c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:49.582870 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984121d8-afa7-4e12-8769-e022acc4c07c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:49.628107 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3109ca69-0f1c-4f6b-bac0-140c7d6f1c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:49.628856 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3109ca69-0f1c-4f6b-bac0-140c7d6f1c88 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:49.629135 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3109ca69-0f1c-4f6b-bac0-140c7d6f1c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:49.629410 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3109ca69-0f1c-4f6b-bac0-140c7d6f1c88 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:49.630641 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3109ca69-0f1c-4f6b-bac0-140c7d6f1c88 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:49.630944 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3109ca69-0f1c-4f6b-bac0-140c7d6f1c88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:49.633557 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3109ca69-0f1c-4f6b-bac0-140c7d6f1c88 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:49.644560 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 489/966] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Jul 13 23:02:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:49.656030 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:49.657813 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3109ca69-0f1c-4f6b-bac0-140c7d6f1c88 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:49.668087 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 478/967] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:02:49.697159 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab9178-1c52-497d-91c6-9f7c5affd2db None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:49.698404 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab9178-1c52-497d-91c6-9f7c5affd2db None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:49.699795 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab9178-1c52-497d-91c6-9f7c5affd2db None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:49.702709 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 479/968] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:49.704499 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab9178-1c52-497d-91c6-9f7c5affd2db None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:49.781094 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:49.793265 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3109ca69-0f1c-4f6b-bac0-140c7d6f1c88 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:49.832837 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e462549-f260-4276-8135-e26338259c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:49.833798 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab9178-1c52-497d-91c6-9f7c5affd2db None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:49.834242 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e462549-f260-4276-8135-e26338259c51 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:49.834350 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e462549-f260-4276-8135-e26338259c51 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:49.834594 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e462549-f260-4276-8135-e26338259c51 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:49.835414 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e462549-f260-4276-8135-e26338259c51 None admin] RBAC: Adding query filter params (interface=admin, service_id=ec2c6db652c345f9a1fc3f87f18533e8, region_id=RegionOne) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:49.835750 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e462549-f260-4276-8135-e26338259c51 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:49.837215 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e462549-f260-4276-8135-e26338259c51 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:49.843309 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 490/969] 10.222.0.60 () {56 vars in 1249 bytes} [Wed Jul 13 23:02:49 2022] GET /identity/v3/endpoints?service_id=ec2c6db652c345f9a1fc3f87f18533e8&interface=admin®ion_id=RegionOne => generated 193 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:49.845202 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5958ad0-b9eb-4569-aa0e-1dc9bd180f0b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:49.845202 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5958ad0-b9eb-4569-aa0e-1dc9bd180f0b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:49.845202 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5958ad0-b9eb-4569-aa0e-1dc9bd180f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:49.867708 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-a5958ad0-b9eb-4569-aa0e-1dc9bd180f0b None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:49.868918 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a5958ad0-b9eb-4569-aa0e-1dc9bd180f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:49.891195 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef87fd3-0bdf-4df0-8d15-a947ad5b912c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:49.891971 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef87fd3-0bdf-4df0-8d15-a947ad5b912c None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:49.892255 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef87fd3-0bdf-4df0-8d15-a947ad5b912c None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:49.892531 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef87fd3-0bdf-4df0-8d15-a947ad5b912c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:49.904041 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef87fd3-0bdf-4df0-8d15-a947ad5b912c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:02:49.912416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-6ef87fd3-0bdf-4df0-8d15-a947ad5b912c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 23:02:49.912416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:49.912416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:49.912416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:49.912416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:49.912416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:49.912416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:49.912416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:49.912416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:49.912416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:49.912416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:49.912416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:49.912416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:02:49.912416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:02:49.912416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:02:49.912416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:02:49.913407 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:02:49.913407 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:02:49.913407 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:02:49.913407 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:02:49.913407 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:49.913407 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:49.913407 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:49.913407 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:49.913407 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:49.913407 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:49.913407 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:49.913407 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:49.913407 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:49.913407 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:49.913407 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:49.913407 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:49.914308 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:49.914308 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:49.914308 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:49.914308 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:49.914308 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:02:49.914308 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:02:49.914308 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:02:49.914308 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:02:49.914308 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:02:49.914308 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:02:49.914308 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 23:02:49.914308 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:49.914308 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 480/970] 10.222.0.60 () {56 vars in 1120 bytes} [Wed Jul 13 23:02:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:49.915016 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef87fd3-0bdf-4df0-8d15-a947ad5b912c None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:49.979980 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 491/971] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:02:49.981492 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5958ad0-b9eb-4569-aa0e-1dc9bd180f0b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:50.000815 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:50.059655 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef87fd3-0bdf-4df0-8d15-a947ad5b912c None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:50.105261 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:50.106067 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bbb180-e3ea-46ab-95e0-77b2deee2f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:50.106985 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bbb180-e3ea-46ab-95e0-77b2deee2f40 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:50.107450 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bbb180-e3ea-46ab-95e0-77b2deee2f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:50.107999 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bbb180-e3ea-46ab-95e0-77b2deee2f40 None admin] PATH_INFO: `/v3/services` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:50.110023 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bbb180-e3ea-46ab-95e0-77b2deee2f40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:50.112398 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bbb180-e3ea-46ab-95e0-77b2deee2f40 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:50.131570 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5958ad0-b9eb-4569-aa0e-1dc9bd180f0b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:50.131802 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 481/972] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Jul 13 23:02:49 2022] GET /identity/v3/services => generated 1497 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:02:50.131802 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bbb180-e3ea-46ab-95e0-77b2deee2f40 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:50.183323 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deee80e-f634-43b5-acfc-d813267c7f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:50.184304 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deee80e-f634-43b5-acfc-d813267c7f8c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:50.184593 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deee80e-f634-43b5-acfc-d813267c7f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:50.184868 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deee80e-f634-43b5-acfc-d813267c7f8c None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:50.185696 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deee80e-f634-43b5-acfc-d813267c7f8c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:50.186353 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deee80e-f634-43b5-acfc-d813267c7f8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:50.188700 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deee80e-f634-43b5-acfc-d813267c7f8c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:50.203091 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 492/973] 10.222.0.60 () {56 vars in 1115 bytes} [Wed Jul 13 23:02:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:50.208977 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deee80e-f634-43b5-acfc-d813267c7f8c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:50.321843 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bbb180-e3ea-46ab-95e0-77b2deee2f40 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:50.348592 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:50.373721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45622976-d995-4339-8d98-3eba59422fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:50.374419 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45622976-d995-4339-8d98-3eba59422fff None admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:50.374698 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45622976-d995-4339-8d98-3eba59422fff None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:50.374970 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45622976-d995-4339-8d98-3eba59422fff None admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:50.375776 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45622976-d995-4339-8d98-3eba59422fff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:50.377289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45622976-d995-4339-8d98-3eba59422fff None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:50.378688 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: WARNING keystone.server.flask.common [None req-45622976-d995-4339-8d98-3eba59422fff None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 13 23:02:50.392940 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.sql.core [None req-45622976-d995-4339-8d98-3eba59422fff None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 13 23:02:50.392940 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 13 23:02:50.392940 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [parameters: {'user_id': '9dd31dd4c0f741cbb37b2ea9f73e386f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 13 23:02:50.392940 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80285) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 13 23:02:50.396911 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deee80e-f634-43b5-acfc-d813267c7f8c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:50.410057 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-45622976-d995-4339-8d98-3eba59422fff None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 13 23:02:50.410057 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:50.410057 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 13 23:02:50.410057 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 13 23:02:50.410057 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 13 23:02:50.410057 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 13 23:02:50.410057 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 13 23:02:50.410057 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application result = self._query(query) Jul 13 23:02:50.410057 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 13 23:02:50.410057 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application conn.query(q) Jul 13 23:02:50.410057 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 13 23:02:50.410057 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 13 23:02:50.410057 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 13 23:02:50.410057 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application result.read() Jul 13 23:02:50.410057 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 13 23:02:50.411054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 13 23:02:50.411054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 13 23:02:50.411054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 13 23:02:50.411054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 13 23:02:50.411054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 13 23:02:50.411054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 13 23:02:50.411054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 13 23:02:50.411054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 13 23:02:50.411054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:50.411054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 13 23:02:50.411054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:50.411054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:50.411054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 13 23:02:50.411054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 13 23:02:50.411054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 13 23:02:50.411054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 13 23:02:50.411054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 13 23:02:50.411054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application next(self.gen) Jul 13 23:02:50.412581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 13 23:02:50.412581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application yield resource Jul 13 23:02:50.412581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 13 23:02:50.412581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application next(self.gen) Jul 13 23:02:50.412581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 13 23:02:50.412581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application self.session.rollback() Jul 13 23:02:50.412581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 13 23:02:50.412581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application self.force_reraise() Jul 13 23:02:50.412581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 13 23:02:50.412581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise self.value Jul 13 23:02:50.412581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 13 23:02:50.412581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 13 23:02:50.412581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 13 23:02:50.412581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application session.commit() Jul 13 23:02:50.412581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 13 23:02:50.412581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 13 23:02:50.412581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 13 23:02:50.413504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application self._prepare_impl() Jul 13 23:02:50.413504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 13 23:02:50.413504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application self.session.flush() Jul 13 23:02:50.413504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 13 23:02:50.413504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application self._flush(objects) Jul 13 23:02:50.413504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 13 23:02:50.413504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 13 23:02:50.413504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 13 23:02:50.413504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application compat.raise_( Jul 13 23:02:50.413504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 13 23:02:50.413504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception Jul 13 23:02:50.413504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 13 23:02:50.413504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application flush_context.execute() Jul 13 23:02:50.413504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 13 23:02:50.413504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rec.execute(self) Jul 13 23:02:50.413504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 13 23:02:50.413504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 13 23:02:50.414398 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 13 23:02:50.414398 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 13 23:02:50.414398 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 13 23:02:50.414398 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 13 23:02:50.414398 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 13 23:02:50.414398 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 13 23:02:50.414398 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 13 23:02:50.414398 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 13 23:02:50.414398 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 13 23:02:50.414398 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 13 23:02:50.414398 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 13 23:02:50.414398 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 13 23:02:50.414398 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 13 23:02:50.414398 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 13 23:02:50.414398 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 13 23:02:50.414398 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception Jul 13 23:02:50.415242 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 13 23:02:50.415242 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 13 23:02:50.415242 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 13 23:02:50.415242 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 13 23:02:50.415242 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 13 23:02:50.415242 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application result = self._query(query) Jul 13 23:02:50.415242 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 13 23:02:50.415242 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application conn.query(q) Jul 13 23:02:50.415242 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 13 23:02:50.415242 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 13 23:02:50.415242 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 13 23:02:50.415242 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application result.read() Jul 13 23:02:50.415242 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 13 23:02:50.415242 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 13 23:02:50.415242 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 13 23:02:50.415242 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 13 23:02:50.418007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 13 23:02:50.418007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 13 23:02:50.418007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 13 23:02:50.418007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 13 23:02:50.418007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 13 23:02:50.418007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 13 23:02:50.418007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [parameters: {'user_id': '9dd31dd4c0f741cbb37b2ea9f73e386f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 13 23:02:50.418007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 13 23:02:50.418007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:50.418007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 23:02:50.418007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:50.418007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:50.418007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:50.418007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:50.418007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:50.420205 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:50.420205 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:50.420205 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:50.420205 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:50.420205 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:50.420205 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:50.420205 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:50.420205 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 13 23:02:50.420205 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 13 23:02:50.420205 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:50.420205 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:50.420205 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 23:02:50.420205 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:50.420205 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 23:02:50.420205 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:50.420205 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 13 23:02:50.420205 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 13 23:02:50.423064 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 13 23:02:50.423064 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 13 23:02:50.423064 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 13 23:02:50.423064 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 13 23:02:50.423064 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 13 23:02:50.423064 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:50.423064 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 482/974] 10.222.0.60 () {60 vars in 1131 bytes} [Wed Jul 13 23:02:50 2022] POST /identity/v3/users => generated 160 bytes in 288 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:50.423064 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc0a18-683c-4fdc-9ed3-2b88e97b19a8 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:50.423064 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc0a18-683c-4fdc-9ed3-2b88e97b19a8 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:50.423064 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc0a18-683c-4fdc-9ed3-2b88e97b19a8 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:50.424350 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 483/975] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:50.424350 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc0a18-683c-4fdc-9ed3-2b88e97b19a8 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:50.458255 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aab097-bdcb-4853-b5d9-725ea8629caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:50.459255 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aab097-bdcb-4853-b5d9-725ea8629caf None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:50.459565 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aab097-bdcb-4853-b5d9-725ea8629caf None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:50.459857 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aab097-bdcb-4853-b5d9-725ea8629caf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:50.475454 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aab097-bdcb-4853-b5d9-725ea8629caf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:50.476970 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aab097-bdcb-4853-b5d9-725ea8629caf None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:50.488733 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:50.502093 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-18aab097-bdcb-4853-b5d9-725ea8629caf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:50.502093 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:50.502093 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:50.502093 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:50.502093 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:50.502093 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:50.502093 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:50.502093 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:50.502093 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:50.502093 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:50.502093 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:50.502093 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:50.502093 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:50.502093 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:50.502093 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:50.503341 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:50.503341 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:50.503341 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:50.503341 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:50.503341 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:50.503341 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:50.503341 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:50.503341 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:50.503341 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:50.503341 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:50.503341 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:50.503341 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:50.503341 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:50.503341 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:50.503341 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:50.503341 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:50.503341 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:50.504722 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:50.504722 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:50.504722 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:50.504722 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:50.508336 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 493/976] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:02:50.510783 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced5a68a-6d1c-466c-8457-762c53e02c75 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:50.511827 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced5a68a-6d1c-466c-8457-762c53e02c75 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:50.512828 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced5a68a-6d1c-466c-8457-762c53e02c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:50.533624 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-ced5a68a-6d1c-466c-8457-762c53e02c75 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:50.534850 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ced5a68a-6d1c-466c-8457-762c53e02c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:50.572433 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc0a18-683c-4fdc-9ed3-2b88e97b19a8 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:50.621810 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a59ad-8593-4e5f-8280-6da0e266c7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:50.622623 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a59ad-8593-4e5f-8280-6da0e266c7dd None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:50.622912 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a59ad-8593-4e5f-8280-6da0e266c7dd None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:50.623243 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a59ad-8593-4e5f-8280-6da0e266c7dd None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:50.632250 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893a59ad-8593-4e5f-8280-6da0e266c7dd None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:50.633913 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893a59ad-8593-4e5f-8280-6da0e266c7dd None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:50.651117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-893a59ad-8593-4e5f-8280-6da0e266c7dd None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 23:02:50.651117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:50.651117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:50.651117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:50.651117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:50.651117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:50.651117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:50.651117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:50.651117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:50.651117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:50.651117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:50.651117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:50.651117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 23:02:50.651117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 23:02:50.651117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 23:02:50.651117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 23:02:50.652157 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:50.652157 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:50.652157 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 23:02:50.652157 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:50.652157 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 23:02:50.652157 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:50.652157 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:50.652157 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:50.652157 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:50.652157 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:50.652157 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:50.652157 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:50.652157 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:50.652157 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:50.652157 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:50.652157 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:50.652157 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:50.652157 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:50.653159 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:50.653159 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:50.653159 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 23:02:50.653159 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 23:02:50.653159 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 23:02:50.653159 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 23:02:50.653159 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 23:02:50.653159 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 23:02:50.653159 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 23:02:50.653159 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:50.653159 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 484/977] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:02:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:50.653159 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a59ad-8593-4e5f-8280-6da0e266c7dd None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:50.663463 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 494/978] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:02:50.664979 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced5a68a-6d1c-466c-8457-762c53e02c75 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:50.714846 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:50.820771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a59ad-8593-4e5f-8280-6da0e266c7dd None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:50.838214 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced5a68a-6d1c-466c-8457-762c53e02c75 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:50.867786 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcedc052-d799-4500-a5af-0e2e0611dcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:50.868623 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcedc052-d799-4500-a5af-0e2e0611dcdf None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:50.868936 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcedc052-d799-4500-a5af-0e2e0611dcdf None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:50.869274 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcedc052-d799-4500-a5af-0e2e0611dcdf None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:50.870094 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcedc052-d799-4500-a5af-0e2e0611dcdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:50.870470 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcedc052-d799-4500-a5af-0e2e0611dcdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:50.872139 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcedc052-d799-4500-a5af-0e2e0611dcdf None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:50.888562 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 485/979] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:50.890907 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcedc052-d799-4500-a5af-0e2e0611dcdf None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:50.897816 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:50.907624 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c78d0e-74ee-4c3e-9183-f674db82868e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:50.908779 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c78d0e-74ee-4c3e-9183-f674db82868e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:50.909265 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c78d0e-74ee-4c3e-9183-f674db82868e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:50.909787 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c78d0e-74ee-4c3e-9183-f674db82868e None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:50.911877 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c78d0e-74ee-4c3e-9183-f674db82868e None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:50.912427 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c78d0e-74ee-4c3e-9183-f674db82868e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:50.915022 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c78d0e-74ee-4c3e-9183-f674db82868e None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:50.991673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 495/980] 10.222.0.60 () {56 vars in 1101 bytes} [Wed Jul 13 23:02:50 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:50.991673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c78d0e-74ee-4c3e-9183-f674db82868e None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:51.076969 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:51.084503 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcedc052-d799-4500-a5af-0e2e0611dcdf None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:51.149839 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29cde32-0c23-4808-96ef-92342068e60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:51.150905 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29cde32-0c23-4808-96ef-92342068e60b None admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:51.151395 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29cde32-0c23-4808-96ef-92342068e60b None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:51.151907 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29cde32-0c23-4808-96ef-92342068e60b None admin] PATH_INFO: `/v3/services` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:51.153325 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29cde32-0c23-4808-96ef-92342068e60b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:51.155962 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29cde32-0c23-4808-96ef-92342068e60b None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:51.187570 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 486/981] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Jul 13 23:02:50 2022] POST /identity/v3/services => generated 242 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:02:51.195679 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c78d0e-74ee-4c3e-9183-f674db82868e None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:51.274581 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd1bf1-c659-4f65-bdc7-6e0674fe306c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:51.275779 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd1bf1-c659-4f65-bdc7-6e0674fe306c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:51.276214 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd1bf1-c659-4f65-bdc7-6e0674fe306c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:51.276640 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd1bf1-c659-4f65-bdc7-6e0674fe306c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:51.290863 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd1bf1-c659-4f65-bdc7-6e0674fe306c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:51.293866 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd1bf1-c659-4f65-bdc7-6e0674fe306c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:51.325117 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-5edd1bf1-c659-4f65-bdc7-6e0674fe306c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 23:02:51.325117 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:51.325117 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:51.325117 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:51.325117 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:51.325117 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:51.325117 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:51.325117 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:51.325117 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:51.325117 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:51.325117 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:51.325117 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:51.325117 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 23:02:51.325117 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 23:02:51.325117 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 23:02:51.325117 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 23:02:51.326423 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:51.326423 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:51.326423 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 23:02:51.326423 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:51.326423 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 23:02:51.326423 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:02:51.326423 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:51.326423 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:51.326423 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:51.326423 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:51.326423 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:51.326423 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:51.326423 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:51.326423 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:51.326423 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:51.326423 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:51.326423 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:51.326423 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:51.328346 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:51.328346 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:51.328346 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 23:02:51.328346 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 23:02:51.328346 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 23:02:51.328346 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 23:02:51.328346 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 23:02:51.328346 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 23:02:51.328346 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 23:02:51.328346 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:51.330972 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 496/982] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Jul 13 23:02:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:51.340090 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29cde32-0c23-4808-96ef-92342068e60b None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:51.515017 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29cde32-0c23-4808-96ef-92342068e60b None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:51.556952 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc5f0f7-d8ce-4b23-b16b-dd294b73b341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:51.557748 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5f0f7-d8ce-4b23-b16b-dd294b73b341 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:51.558170 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5f0f7-d8ce-4b23-b16b-dd294b73b341 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:51.558580 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5f0f7-d8ce-4b23-b16b-dd294b73b341 None admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:51.559662 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc5f0f7-d8ce-4b23-b16b-dd294b73b341 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:51.560112 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc5f0f7-d8ce-4b23-b16b-dd294b73b341 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:51.561863 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc5f0f7-d8ce-4b23-b16b-dd294b73b341 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:51.589023 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 487/983] 10.222.0.60 () {56 vars in 1149 bytes} [Wed Jul 13 23:02:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:51.591060 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd1bf1-c659-4f65-bdc7-6e0674fe306c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:51.649405 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:51.722887 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd1bf1-c659-4f65-bdc7-6e0674fe306c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:51.768277 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40265ac8-155a-4cf8-ac37-2cf3f49dea13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:51.768952 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40265ac8-155a-4cf8-ac37-2cf3f49dea13 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:51.769230 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40265ac8-155a-4cf8-ac37-2cf3f49dea13 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:51.769506 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40265ac8-155a-4cf8-ac37-2cf3f49dea13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:51.859423 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40265ac8-155a-4cf8-ac37-2cf3f49dea13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:51.860884 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40265ac8-155a-4cf8-ac37-2cf3f49dea13 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:51.918078 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:51.999649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 497/984] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:02:51 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 13 23:02:52.001925 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc5f0f7-d8ce-4b23-b16b-dd294b73b341 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:52.037119 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:52.185678 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc5f0f7-d8ce-4b23-b16b-dd294b73b341 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:52.254449 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67ae69-39cf-44c2-9bf2-f0443c7c6201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:52.255870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67ae69-39cf-44c2-9bf2-f0443c7c6201 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:52.256382 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67ae69-39cf-44c2-9bf2-f0443c7c6201 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:52.256878 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67ae69-39cf-44c2-9bf2-f0443c7c6201 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:52.277385 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc67ae69-39cf-44c2-9bf2-f0443c7c6201 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:52.280165 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc67ae69-39cf-44c2-9bf2-f0443c7c6201 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:52.315337 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-fc67ae69-39cf-44c2-9bf2-f0443c7c6201 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:52.315337 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:52.315337 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:52.315337 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:52.315337 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:52.315337 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:52.315337 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:52.315337 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:52.315337 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:52.315337 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:52.315337 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:52.315337 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:52.315337 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:02:52.315337 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:02:52.315337 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:02:52.317144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:02:52.317144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:52.317144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:52.317144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:52.317144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:52.317144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:52.317144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:52.317144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:52.317144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:52.317144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:52.317144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:52.317144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:52.317144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:52.317144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:52.317144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:52.317144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:52.317144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:52.325673 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:02:52.325673 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:02:52.325673 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:02:52.325673 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:52.325673 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 488/985] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:02:52.330294 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40265ac8-155a-4cf8-ac37-2cf3f49dea13 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:52.444520 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844b89b-0cd0-48b7-9e95-440e127a614f None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:52.444698 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844b89b-0cd0-48b7-9e95-440e127a614f None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:52.445351 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844b89b-0cd0-48b7-9e95-440e127a614f None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:52.450166 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 489/986] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:52.451911 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073848ed-a24d-47b5-a5ff-9cad6c17653a None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:52.452332 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073848ed-a24d-47b5-a5ff-9cad6c17653a None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:52.452607 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073848ed-a24d-47b5-a5ff-9cad6c17653a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:52.473086 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-073848ed-a24d-47b5-a5ff-9cad6c17653a None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:52.475198 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-073848ed-a24d-47b5-a5ff-9cad6c17653a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:52.505765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40265ac8-155a-4cf8-ac37-2cf3f49dea13 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:52.541430 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:52.555700 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 490/987] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:52 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:02:52.560917 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6f5be-97c4-45fb-b2d2-24e6d0dbe2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:52.561610 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6f5be-97c4-45fb-b2d2-24e6d0dbe2d5 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:52.561896 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6f5be-97c4-45fb-b2d2-24e6d0dbe2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:52.562168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6f5be-97c4-45fb-b2d2-24e6d0dbe2d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:52.562898 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6f5be-97c4-45fb-b2d2-24e6d0dbe2d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:52.563199 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6f5be-97c4-45fb-b2d2-24e6d0dbe2d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:52.564636 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6f5be-97c4-45fb-b2d2-24e6d0dbe2d5 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:52.572568 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073848ed-a24d-47b5-a5ff-9cad6c17653a None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:52.579580 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 498/988] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:02:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:52.589312 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6f5be-97c4-45fb-b2d2-24e6d0dbe2d5 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:52.595157 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:52.719035 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073848ed-a24d-47b5-a5ff-9cad6c17653a None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:52.761623 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df827c-75ef-40ab-901b-2f48cda5a90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:52.761837 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6f5be-97c4-45fb-b2d2-24e6d0dbe2d5 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:52.762937 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df827c-75ef-40ab-901b-2f48cda5a90b None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:52.763416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df827c-75ef-40ab-901b-2f48cda5a90b None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:52.763918 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df827c-75ef-40ab-901b-2f48cda5a90b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:52.764815 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9df827c-75ef-40ab-901b-2f48cda5a90b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:52.766541 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9df827c-75ef-40ab-901b-2f48cda5a90b None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:52.779641 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-c9df827c-75ef-40ab-901b-2f48cda5a90b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 23:02:52.779641 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:52.779641 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 23:02:52.779641 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 23:02:52.779641 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 23:02:52.779641 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 23:02:52.779641 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 23:02:52.779641 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:52.779641 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 23:02:52.779641 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:52.779641 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 23:02:52.779641 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:52.779641 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:52.779641 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:52.779641 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:52.779641 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:52.779641 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:52.780716 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:52.780716 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:52.780716 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:52.780716 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:52.780716 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:52.780716 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:52.780716 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 23:02:52.780716 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 23:02:52.780716 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 23:02:52.780716 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 23:02:52.780716 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:52.780716 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:52.780716 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:52.780716 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:52.780716 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:52.780716 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:52.781585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:52.781585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:52.781585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:52.781585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:52.781585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:52.781585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:52.781585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:52.781585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:52.781585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:52.781585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:52.781585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 23:02:52.781585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:52.781585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 23:02:52.781585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:02:52.783874 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 491/989] 10.222.0.60 () {56 vars in 1126 bytes} [Wed Jul 13 23:02:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:52.787004 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df827c-75ef-40ab-901b-2f48cda5a90b None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:52.818029 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d51d30a-2ab3-42a6-8bfe-4d7114f04624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:52.819204 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d51d30a-2ab3-42a6-8bfe-4d7114f04624 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:52.819693 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d51d30a-2ab3-42a6-8bfe-4d7114f04624 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:52.820390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d51d30a-2ab3-42a6-8bfe-4d7114f04624 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:52.827834 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d51d30a-2ab3-42a6-8bfe-4d7114f04624 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:52.829595 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d51d30a-2ab3-42a6-8bfe-4d7114f04624 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:52.845240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-5d51d30a-2ab3-42a6-8bfe-4d7114f04624 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:52.845240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:52.845240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:52.845240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:52.845240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:52.845240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:52.845240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:52.845240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:52.845240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:52.845240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:52.845240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:52.845240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:52.845240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 23:02:52.845240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 23:02:52.845240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 23:02:52.846191 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 23:02:52.846191 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:52.846191 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:52.846191 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:52.846191 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:52.846191 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:52.846191 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:52.846191 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:52.846191 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:52.846191 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:52.846191 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:52.846191 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:52.846191 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:52.846191 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:52.846191 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:52.846191 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:52.846191 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:52.847086 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 23:02:52.847086 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 23:02:52.847086 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 23:02:52.847086 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 23:02:52.847086 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 23:02:52.847086 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 23:02:52.847086 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:02:52.847086 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:52.849206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 499/990] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:02:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:02:52.854982 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d51d30a-2ab3-42a6-8bfe-4d7114f04624 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:52.884659 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:52.954071 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df827c-75ef-40ab-901b-2f48cda5a90b None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:52.957832 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:53.013321 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b046d7f3-77b6-4da0-a661-b2e7ceae6da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:53.014486 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b046d7f3-77b6-4da0-a661-b2e7ceae6da0 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:53.014947 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b046d7f3-77b6-4da0-a661-b2e7ceae6da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:53.015853 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b046d7f3-77b6-4da0-a661-b2e7ceae6da0 None admin] PATH_INFO: `/v3/services` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:53.017067 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b046d7f3-77b6-4da0-a661-b2e7ceae6da0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:53.017573 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b046d7f3-77b6-4da0-a661-b2e7ceae6da0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:53.019994 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b046d7f3-77b6-4da0-a661-b2e7ceae6da0 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:53.035567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 492/991] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Jul 13 23:02:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:02:53.035845 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d51d30a-2ab3-42a6-8bfe-4d7114f04624 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:53.040318 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b046d7f3-77b6-4da0-a661-b2e7ceae6da0 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:53.078902 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e28880-b1b8-4bfe-b935-a2100e5d7719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:53.079694 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e28880-b1b8-4bfe-b935-a2100e5d7719 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:53.080046 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e28880-b1b8-4bfe-b935-a2100e5d7719 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:53.080379 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e28880-b1b8-4bfe-b935-a2100e5d7719 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:53.081260 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e28880-b1b8-4bfe-b935-a2100e5d7719 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:53.081719 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e28880-b1b8-4bfe-b935-a2100e5d7719 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:53.083941 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e28880-b1b8-4bfe-b935-a2100e5d7719 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:53.099583 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 500/992] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jul 13 23:02:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:02:53.107827 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e28880-b1b8-4bfe-b935-a2100e5d7719 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:53.233342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e28880-b1b8-4bfe-b935-a2100e5d7719 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:53.234314 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b046d7f3-77b6-4da0-a661-b2e7ceae6da0 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:53.264703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:53.279694 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:53.290687 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac2426-f030-4b40-952e-df641cf6305d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:53.291824 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac2426-f030-4b40-952e-df641cf6305d None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:53.292233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac2426-f030-4b40-952e-df641cf6305d None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:53.292676 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac2426-f030-4b40-952e-df641cf6305d None admin] PATH_INFO: `/v3/services` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:53.293541 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac2426-f030-4b40-952e-df641cf6305d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:53.293701 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218832f2-6a0b-40b4-a744-b4308e9bd34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:53.294044 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac2426-f030-4b40-952e-df641cf6305d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:53.294751 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218832f2-6a0b-40b4-a744-b4308e9bd34b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:53.295205 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218832f2-6a0b-40b4-a744-b4308e9bd34b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:53.295610 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218832f2-6a0b-40b4-a744-b4308e9bd34b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:53.296003 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac2426-f030-4b40-952e-df641cf6305d None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:53.297179 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218832f2-6a0b-40b4-a744-b4308e9bd34b None admin] RBAC: Adding query filter params (role.id=f7411bbcd41a4f959f76b130485bfb89, scope.project.id=ab0a5581e4e444c293815e1304e20c05, user.id=a32702a67a7b49ffa153fc59251fd169) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:53.297755 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218832f2-6a0b-40b4-a744-b4308e9bd34b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:53.300619 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218832f2-6a0b-40b4-a744-b4308e9bd34b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:53.308013 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 493/993] 10.222.0.60 () {56 vars in 1122 bytes} [Wed Jul 13 23:02:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:02:53.309184 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac2426-f030-4b40-952e-df641cf6305d None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:53.317067 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 501/994] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:02:53 2022] GET /identity/v3/role_assignments?user.id=a32702a67a7b49ffa153fc59251fd169&scope.project.id=ab0a5581e4e444c293815e1304e20c05&role.id=f7411bbcd41a4f959f76b130485bfb89 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:02:53.511176 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac2426-f030-4b40-952e-df641cf6305d None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:53.577227 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0849ab0-8026-4e61-bf09-e4730f85a017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:53.578223 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0849ab0-8026-4e61-bf09-e4730f85a017 None admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:53.578713 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0849ab0-8026-4e61-bf09-e4730f85a017 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:53.579201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0849ab0-8026-4e61-bf09-e4730f85a017 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:53.580719 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0849ab0-8026-4e61-bf09-e4730f85a017 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:53.583934 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0849ab0-8026-4e61-bf09-e4730f85a017 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:53.611610 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 494/995] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Jul 13 23:02:53 2022] POST /identity/v3/endpoints => generated 325 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:02:53.747896 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d98229f-a776-472d-9f51-76719b4688a8 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:53.748232 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d98229f-a776-472d-9f51-76719b4688a8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:53.748658 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d98229f-a776-472d-9f51-76719b4688a8 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:53.750187 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 502/996] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:02:53.756410 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfcf8a-9001-483d-88e2-504698c9d6fb None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:53.756811 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfcf8a-9001-483d-88e2-504698c9d6fb None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:53.756971 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfcf8a-9001-483d-88e2-504698c9d6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:53.782846 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-2dbfcf8a-9001-483d-88e2-504698c9d6fb None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:53.785577 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-2dbfcf8a-9001-483d-88e2-504698c9d6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:53.858586 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:53.894600 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 495/997] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:53 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:02:53.916887 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d98229f-a776-472d-9f51-76719b4688a8 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:54.024820 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d98229f-a776-472d-9f51-76719b4688a8 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:54.064292 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24610620-7647-4388-aa2a-af789718c536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:54.065098 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24610620-7647-4388-aa2a-af789718c536 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:54.065378 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24610620-7647-4388-aa2a-af789718c536 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:54.065649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24610620-7647-4388-aa2a-af789718c536 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:54.066298 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24610620-7647-4388-aa2a-af789718c536 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:54.067773 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24610620-7647-4388-aa2a-af789718c536 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:54.081696 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-24610620-7647-4388-aa2a-af789718c536 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 23:02:54.081696 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:54.081696 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 23:02:54.081696 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 23:02:54.081696 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 23:02:54.081696 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 23:02:54.081696 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 23:02:54.081696 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:54.081696 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 23:02:54.081696 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:54.081696 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 23:02:54.081696 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:54.081696 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:54.081696 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:54.081696 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:54.081696 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:54.081696 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:54.083924 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:54.083924 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:54.083924 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:54.083924 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:54.083924 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:54.083924 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:54.083924 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 23:02:54.083924 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 23:02:54.083924 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 23:02:54.083924 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 23:02:54.083924 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:54.083924 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:54.083924 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:54.083924 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:54.083924 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:54.083924 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:54.085622 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:54.085622 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:54.085622 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:54.085622 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:54.085622 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:54.085622 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:54.085622 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:54.085622 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:54.085622 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:54.085622 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:54.085622 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 23:02:54.085622 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:54.085622 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 23:02:54.085622 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:54.093680 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 503/998] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:02:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:54.094025 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbfcf8a-9001-483d-88e2-504698c9d6fb None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:54.274248 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbfcf8a-9001-483d-88e2-504698c9d6fb None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:54.335486 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e8ad5d-8aff-4824-8281-e239b37b451a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:54.336649 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e8ad5d-8aff-4824-8281-e239b37b451a None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:54.337142 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e8ad5d-8aff-4824-8281-e239b37b451a None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:54.337611 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e8ad5d-8aff-4824-8281-e239b37b451a None admin] PATH_INFO: `/v3/services` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:54.338870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e8ad5d-8aff-4824-8281-e239b37b451a None admin] RBAC: Adding query filter params (name=compute) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:54.339409 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e8ad5d-8aff-4824-8281-e239b37b451a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:54.342015 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e8ad5d-8aff-4824-8281-e239b37b451a None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:54.355784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 496/999] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Jul 13 23:02:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:54.357979 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24610620-7647-4388-aa2a-af789718c536 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:54.466819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:54.559506 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24610620-7647-4388-aa2a-af789718c536 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:54.623644 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876e1b95-6f6d-4023-8d12-629d2bab7197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:54.624780 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e1b95-6f6d-4023-8d12-629d2bab7197 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:54.625271 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e1b95-6f6d-4023-8d12-629d2bab7197 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:54.625774 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e1b95-6f6d-4023-8d12-629d2bab7197 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:54.627143 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876e1b95-6f6d-4023-8d12-629d2bab7197 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:54.627739 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876e1b95-6f6d-4023-8d12-629d2bab7197 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:54.630404 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876e1b95-6f6d-4023-8d12-629d2bab7197 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:54.647635 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 504/1000] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Jul 13 23:02:54 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:54.650110 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e8ad5d-8aff-4824-8281-e239b37b451a None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:54.726321 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:54.826979 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e8ad5d-8aff-4824-8281-e239b37b451a None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:54.890480 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c6e39-b998-4dba-908b-363fb3675b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:54.891648 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c6e39-b998-4dba-908b-363fb3675b57 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:54.892127 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c6e39-b998-4dba-908b-363fb3675b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:54.892608 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c6e39-b998-4dba-908b-363fb3675b57 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:54.894159 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c6e39-b998-4dba-908b-363fb3675b57 None admin] RBAC: Adding query filter params (interface=public, service_id=c20726e9bdc448f1bd31d18b3a09c9d5, region_id=RegionOne) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:54.894715 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c6e39-b998-4dba-908b-363fb3675b57 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:54.897924 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c6e39-b998-4dba-908b-363fb3675b57 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:54.908577 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 497/1001] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Jul 13 23:02:54 2022] GET /identity/v3/endpoints?service_id=c20726e9bdc448f1bd31d18b3a09c9d5&interface=public®ion_id=RegionOne => generated 194 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:02:54.917554 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876e1b95-6f6d-4023-8d12-629d2bab7197 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:55.048682 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876e1b95-6f6d-4023-8d12-629d2bab7197 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:55.068139 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:55.090194 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda94d8b-7abc-4a3a-acef-ab511e97e059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:55.090753 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda94d8b-7abc-4a3a-acef-ab511e97e059 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:55.091069 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda94d8b-7abc-4a3a-acef-ab511e97e059 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:55.091352 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda94d8b-7abc-4a3a-acef-ab511e97e059 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:55.092081 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda94d8b-7abc-4a3a-acef-ab511e97e059 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:55.093477 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda94d8b-7abc-4a3a-acef-ab511e97e059 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:55.107138 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 505/1002] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Jul 13 23:02:54 2022] GET /identity/v3/services => generated 1727 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:02:55.941506 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73373e0-9ec7-4cfe-99d4-b01a0a535145 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:55.942351 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73373e0-9ec7-4cfe-99d4-b01a0a535145 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:55.942351 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73373e0-9ec7-4cfe-99d4-b01a0a535145 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:55.943906 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 498/1003] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:55.953102 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc5c30-cdb9-48d8-aa78-6f0eb0da5097 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:55.953565 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc5c30-cdb9-48d8-aa78-6f0eb0da5097 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:55.954023 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc5c30-cdb9-48d8-aa78-6f0eb0da5097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:55.983686 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-cebc5c30-cdb9-48d8-aa78-6f0eb0da5097 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:55.985583 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-cebc5c30-cdb9-48d8-aa78-6f0eb0da5097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:56.107601 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 506/1004] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:55 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:02:56.130269 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73373e0-9ec7-4cfe-99d4-b01a0a535145 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:56.310668 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73373e0-9ec7-4cfe-99d4-b01a0a535145 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:56.322680 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:56.333885 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb110780-a0c9-4e82-837a-6ba66d80bec8 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:56.334378 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb110780-a0c9-4e82-837a-6ba66d80bec8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:56.335368 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb110780-a0c9-4e82-837a-6ba66d80bec8 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:56.338281 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 507/1005] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:02:56.345063 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807837f-0268-4b5d-b316-7eee419b3c8e None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:56.345541 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807837f-0268-4b5d-b316-7eee419b3c8e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:56.346033 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807837f-0268-4b5d-b316-7eee419b3c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:56.369594 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8de578-731b-42ff-8003-0daa4f49e4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:56.371199 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8de578-731b-42ff-8003-0daa4f49e4bb None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:56.371502 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8de578-731b-42ff-8003-0daa4f49e4bb None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:56.371796 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8de578-731b-42ff-8003-0daa4f49e4bb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:56.373180 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8de578-731b-42ff-8003-0daa4f49e4bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:56.375134 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8de578-731b-42ff-8003-0daa4f49e4bb None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:56.377299 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 499/1006] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:56.378904 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-6807837f-0268-4b5d-b316-7eee419b3c8e None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:56.381420 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6807837f-0268-4b5d-b316-7eee419b3c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:56.383657 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8de578-731b-42ff-8003-0daa4f49e4bb None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:56.508163 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 508/1007] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:56 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:02:56.533130 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807837f-0268-4b5d-b316-7eee419b3c8e None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:56.539862 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8de578-731b-42ff-8003-0daa4f49e4bb None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:56.581164 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce507af-e887-49cb-a0c6-5874371ec129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:56.582184 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce507af-e887-49cb-a0c6-5874371ec129 None admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:56.582617 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce507af-e887-49cb-a0c6-5874371ec129 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:56.583991 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce507af-e887-49cb-a0c6-5874371ec129 None admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:56.584841 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce507af-e887-49cb-a0c6-5874371ec129 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:56.586386 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce507af-e887-49cb-a0c6-5874371ec129 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:56.610316 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:56.621821 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 500/1008] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Jul 13 23:02:56 2022] POST /identity/v3/projects => generated 311 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:02:56.741256 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807837f-0268-4b5d-b316-7eee419b3c8e None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:56.813650 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1a2237-53e0-4740-970d-d8c9522f0c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:56.815028 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1a2237-53e0-4740-970d-d8c9522f0c44 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:56.815527 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1a2237-53e0-4740-970d-d8c9522f0c44 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:56.816028 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1a2237-53e0-4740-970d-d8c9522f0c44 None admin] PATH_INFO: `/v3/services/image` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:56.817266 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1a2237-53e0-4740-970d-d8c9522f0c44 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:56.820686 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1a2237-53e0-4740-970d-d8c9522f0c44 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:56.835930 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-cc1a2237-53e0-4740-970d-d8c9522f0c44 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 23:02:56.835930 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:56.835930 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 23:02:56.835930 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 23:02:56.835930 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 23:02:56.835930 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 23:02:56.835930 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 23:02:56.835930 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:56.835930 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 23:02:56.835930 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:56.835930 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 23:02:56.835930 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:56.835930 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:56.835930 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:56.835930 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:56.835930 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:56.835930 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:56.837710 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:56.837710 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:56.837710 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:56.837710 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:56.837710 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:56.837710 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:56.837710 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 23:02:56.837710 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 23:02:56.837710 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 23:02:56.837710 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 23:02:56.837710 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:56.837710 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:56.837710 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:56.837710 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:56.837710 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:56.837710 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:56.840168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:56.840168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:56.840168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:56.840168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:56.840168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:56.840168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:56.840168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:56.840168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:56.840168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:56.840168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:56.840168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 23:02:56.840168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:56.840168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 23:02:56.840168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:56.840168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 509/1009] 10.222.0.60 () {56 vars in 1105 bytes} [Wed Jul 13 23:02:56 2022] GET /identity/v3/services/image => generated 86 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:02:56.849290 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce507af-e887-49cb-a0c6-5874371ec129 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:56.903725 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:57.010521 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce507af-e887-49cb-a0c6-5874371ec129 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:57.049039 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49047b-e301-413c-8b1e-8d75bcf1e292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:57.049709 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49047b-e301-413c-8b1e-8d75bcf1e292 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:57.049989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49047b-e301-413c-8b1e-8d75bcf1e292 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:57.050255 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49047b-e301-413c-8b1e-8d75bcf1e292 None admin] PATH_INFO: `/v3/services` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:57.050970 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d49047b-e301-413c-8b1e-8d75bcf1e292 None admin] RBAC: Adding query filter params (name=image) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:57.051267 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d49047b-e301-413c-8b1e-8d75bcf1e292 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:57.053330 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d49047b-e301-413c-8b1e-8d75bcf1e292 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:57.059348 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 501/1010] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:57.065863 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1a2237-53e0-4740-970d-d8c9522f0c44 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:57.254572 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1a2237-53e0-4740-970d-d8c9522f0c44 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:57.258932 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:57.307436 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851feb4-ac22-437a-8ad7-7edabbf3f60f None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:57.307760 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851feb4-ac22-437a-8ad7-7edabbf3f60f None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:57.308053 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851feb4-ac22-437a-8ad7-7edabbf3f60f None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:57.309451 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 502/1011] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:02:57.314934 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7017a-372a-4dcc-9b6c-faef0b5173c1 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:57.315204 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7017a-372a-4dcc-9b6c-faef0b5173c1 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:57.315590 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d7017a-372a-4dcc-9b6c-faef0b5173c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:57.317769 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2edba9-afc6-4ec2-8aa5-d55b5234dd18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:57.318436 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2edba9-afc6-4ec2-8aa5-d55b5234dd18 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:57.318714 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2edba9-afc6-4ec2-8aa5-d55b5234dd18 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:57.318988 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2edba9-afc6-4ec2-8aa5-d55b5234dd18 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:57.320008 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2edba9-afc6-4ec2-8aa5-d55b5234dd18 None admin] RBAC: Adding query filter params (type=image) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:57.320285 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2edba9-afc6-4ec2-8aa5-d55b5234dd18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:57.321670 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2edba9-afc6-4ec2-8aa5-d55b5234dd18 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:57.335649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 510/1012] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:57 2022] GET /identity/v3/services?type=image => generated 122 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:02:57.342755 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-07d7017a-372a-4dcc-9b6c-faef0b5173c1 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:57.344047 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-07d7017a-372a-4dcc-9b6c-faef0b5173c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:57.474206 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 503/1013] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:57 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 23:02:57.497534 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2edba9-afc6-4ec2-8aa5-d55b5234dd18 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:57.646350 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2edba9-afc6-4ec2-8aa5-d55b5234dd18 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:57.685187 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf4a314-4734-4d76-a4a4-edd0b3394afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:57.685986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf4a314-4734-4d76-a4a4-edd0b3394afe None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:57.686260 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf4a314-4734-4d76-a4a4-edd0b3394afe None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:57.686537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf4a314-4734-4d76-a4a4-edd0b3394afe None admin] PATH_INFO: `/v3/services/compute` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:57.687203 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf4a314-4734-4d76-a4a4-edd0b3394afe None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:57.688638 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf4a314-4734-4d76-a4a4-edd0b3394afe None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:57.701819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-ebf4a314-4734-4d76-a4a4-edd0b3394afe None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 23:02:57.701819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:57.701819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 23:02:57.701819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 23:02:57.701819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 23:02:57.701819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 23:02:57.701819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 23:02:57.701819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:57.701819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 23:02:57.701819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:57.701819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 23:02:57.701819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:57.701819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:02:57.701819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:02:57.701819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:02:57.701819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:02:57.701819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:02:57.703914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:02:57.703914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:02:57.703914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:02:57.703914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:02:57.703914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:02:57.703914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:02:57.703914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 23:02:57.703914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 23:02:57.703914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 23:02:57.703914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 23:02:57.703914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:02:57.703914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:02:57.703914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:02:57.703914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:02:57.703914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:02:57.703914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:02:57.705683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:02:57.705683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:02:57.705683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:02:57.705683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:02:57.705683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:02:57.705683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:02:57.705683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:02:57.705683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:02:57.705683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:02:57.705683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:02:57.705683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 23:02:57.705683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:02:57.705683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 23:02:57.705683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:02:57.705683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 511/1014] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:02:57 2022] GET /identity/v3/services/compute => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:02:57.710119 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d7017a-372a-4dcc-9b6c-faef0b5173c1 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:57.733736 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:57.821386 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d7017a-372a-4dcc-9b6c-faef0b5173c1 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:57.883909 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd99e0c-e851-4664-88cd-8476289fc532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:57.885072 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd99e0c-e851-4664-88cd-8476289fc532 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:57.885576 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd99e0c-e851-4664-88cd-8476289fc532 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:57.886071 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd99e0c-e851-4664-88cd-8476289fc532 None admin] PATH_INFO: `/v3/services` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:57.887618 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd99e0c-e851-4664-88cd-8476289fc532 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:57.888121 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd99e0c-e851-4664-88cd-8476289fc532 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:57.890744 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd99e0c-e851-4664-88cd-8476289fc532 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:57.905608 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 504/1015] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Jul 13 23:02:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:57.907581 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf4a314-4734-4d76-a4a4-edd0b3394afe None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:57.967842 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:58.066499 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf4a314-4734-4d76-a4a4-edd0b3394afe None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:58.110365 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1765d7-cb3c-48f9-9971-f846760f1619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:58.111142 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1765d7-cb3c-48f9-9971-f846760f1619 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:58.111433 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1765d7-cb3c-48f9-9971-f846760f1619 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:58.111757 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1765d7-cb3c-48f9-9971-f846760f1619 None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:58.112506 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1765d7-cb3c-48f9-9971-f846760f1619 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:02:58.112814 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1765d7-cb3c-48f9-9971-f846760f1619 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:58.114786 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1765d7-cb3c-48f9-9971-f846760f1619 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:58.121022 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 512/1016] 10.222.0.60 () {56 vars in 1112 bytes} [Wed Jul 13 23:02:57 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:58.126719 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd99e0c-e851-4664-88cd-8476289fc532 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:58.224968 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:58.245069 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd99e0c-e851-4664-88cd-8476289fc532 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:58.283801 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e53e75a-1fc0-43db-9b89-becf3a74783d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:58.284485 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e53e75a-1fc0-43db-9b89-becf3a74783d None admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:58.284771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e53e75a-1fc0-43db-9b89-becf3a74783d None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:58.285048 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e53e75a-1fc0-43db-9b89-becf3a74783d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:58.285897 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e53e75a-1fc0-43db-9b89-becf3a74783d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:58.287397 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e53e75a-1fc0-43db-9b89-becf3a74783d None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:58.309039 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 505/1017] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Jul 13 23:02:58 2022] POST /identity/v3/endpoints => generated 334 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:02:59.405737 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fad03a-5863-420e-9642-808e72f7dee2 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:59.406285 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fad03a-5863-420e-9642-808e72f7dee2 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:59.406285 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fad03a-5863-420e-9642-808e72f7dee2 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:59.407772 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 513/1018] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:02:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:02:59.414300 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac64251-23b2-4d21-b93c-311615fd79fb None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:59.414864 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac64251-23b2-4d21-b93c-311615fd79fb None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:59.414864 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac64251-23b2-4d21-b93c-311615fd79fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:59.433616 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-5ac64251-23b2-4d21-b93c-311615fd79fb None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:02:59.434874 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-5ac64251-23b2-4d21-b93c-311615fd79fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:02:59.514270 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 506/1019] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:02:59 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:02:59.540536 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fad03a-5863-420e-9642-808e72f7dee2 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:59.694622 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:59.712382 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fad03a-5863-420e-9642-808e72f7dee2 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:59.780435 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c27e72-3a28-48db-9015-04d17e020c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:02:59.781236 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c27e72-3a28-48db-9015-04d17e020c32 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:02:59.781513 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c27e72-3a28-48db-9015-04d17e020c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:02:59.781776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c27e72-3a28-48db-9015-04d17e020c32 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:02:59.783702 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c27e72-3a28-48db-9015-04d17e020c32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:02:59.784482 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c27e72-3a28-48db-9015-04d17e020c32 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:02:59.786916 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 514/1020] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:02:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:02:59.792568 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac64251-23b2-4d21-b93c-311615fd79fb None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:02:59.870725 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:02:59.934859 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac64251-23b2-4d21-b93c-311615fd79fb None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:02:59.999597 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94493a-f9fb-45c1-88b2-3feb6ab998dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:00.000461 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94493a-f9fb-45c1-88b2-3feb6ab998dd None admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:00.000872 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94493a-f9fb-45c1-88b2-3feb6ab998dd None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:00.001294 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94493a-f9fb-45c1-88b2-3feb6ab998dd None admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:00.002601 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e94493a-f9fb-45c1-88b2-3feb6ab998dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:00.005936 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e94493a-f9fb-45c1-88b2-3feb6ab998dd None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:00.043398 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 507/1021] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Jul 13 23:02:59 2022] POST /identity/v3/users => generated 285 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:03:00.496814 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae9a1f-5df4-4961-b77f-9b752c3b441e None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:00.497381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae9a1f-5df4-4961-b77f-9b752c3b441e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:00.497796 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae9a1f-5df4-4961-b77f-9b752c3b441e None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:00.500495 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 515/1022] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:03:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:03:00.507005 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fa775-256e-48b4-a45e-5aa38ea48ccf None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:00.507819 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fa775-256e-48b4-a45e-5aa38ea48ccf None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:00.507819 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7fa775-256e-48b4-a45e-5aa38ea48ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:00.512083 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e53fa-5a82-4df9-8219-2376b0cd35c4 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:00.512434 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e53fa-5a82-4df9-8219-2376b0cd35c4 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:00.512672 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8e53fa-5a82-4df9-8219-2376b0cd35c4 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:00.514298 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 516/1023] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:03:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:03:00.519996 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a6dc1-599b-4bc3-a8ad-d6b78057677f None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:00.520423 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a6dc1-599b-4bc3-a8ad-d6b78057677f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:00.520697 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a6dc1-599b-4bc3-a8ad-d6b78057677f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:00.527422 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-cf7fa775-256e-48b4-a45e-5aa38ea48ccf None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:03:00.528771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-cf7fa775-256e-48b4-a45e-5aa38ea48ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:03:00.539153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-038a6dc1-599b-4bc3-a8ad-d6b78057677f None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:03:00.540746 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-038a6dc1-599b-4bc3-a8ad-d6b78057677f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:03:00.607445 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 508/1024] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:03:00 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 23:03:00.631288 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7fa775-256e-48b4-a45e-5aa38ea48ccf None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:00.654924 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 517/1025] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:03:00 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 23:03:00.694871 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:00.698587 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a6dc1-599b-4bc3-a8ad-d6b78057677f None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:00.781998 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7fa775-256e-48b4-a45e-5aa38ea48ccf None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:00.816804 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:00.822908 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6701fd1-85e5-42c8-9bc5-be355b6232d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:00.823655 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6701fd1-85e5-42c8-9bc5-be355b6232d8 None admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:00.823944 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6701fd1-85e5-42c8-9bc5-be355b6232d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:00.824374 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6701fd1-85e5-42c8-9bc5-be355b6232d8 None admin] PATH_INFO: `/v3/services` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:00.825113 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6701fd1-85e5-42c8-9bc5-be355b6232d8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:00.826533 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6701fd1-85e5-42c8-9bc5-be355b6232d8 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:00.831319 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a6dc1-599b-4bc3-a8ad-d6b78057677f None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:00.847006 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 509/1026] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Jul 13 23:03:00 2022] POST /identity/v3/services => generated 242 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:03:00.912397 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df883c-49f8-4922-84a3-3bd6dbe88a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:00.913477 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df883c-49f8-4922-84a3-3bd6dbe88a13 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:00.913904 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df883c-49f8-4922-84a3-3bd6dbe88a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:00.914329 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df883c-49f8-4922-84a3-3bd6dbe88a13 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:00.933011 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7df883c-49f8-4922-84a3-3bd6dbe88a13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:03:00.949237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-a7df883c-49f8-4922-84a3-3bd6dbe88a13 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 23:03:00.949237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:03:00.949237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:03:00.949237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:03:00.949237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:03:00.949237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:03:00.949237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:03:00.949237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:03:00.949237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:03:00.949237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:03:00.949237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:03:00.949237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:03:00.949237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:03:00.949237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:03:00.949237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:03:00.949237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:03:00.950380 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:03:00.950380 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:03:00.950380 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:03:00.950380 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:03:00.950380 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:03:00.950380 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:03:00.950380 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:03:00.950380 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:03:00.950380 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:03:00.950380 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:03:00.950380 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:03:00.950380 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:03:00.950380 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:03:00.950380 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:03:00.950380 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:03:00.950380 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:03:00.951305 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:03:00.951305 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:03:00.951305 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:03:00.951305 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:03:00.951305 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:03:00.951305 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:03:00.951305 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:03:00.951305 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:03:00.951305 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:03:00.951305 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:03:00.951305 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 23:03:00.951305 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:03:00.959873 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 518/1027] 10.222.0.60 () {56 vars in 1120 bytes} [Wed Jul 13 23:03:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:03:00.961330 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6701fd1-85e5-42c8-9bc5-be355b6232d8 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:01.106150 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6701fd1-85e5-42c8-9bc5-be355b6232d8 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:01.131058 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:01.171431 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86dab4-eca8-4af7-80ea-68de2f2c66e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:01.172923 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86dab4-eca8-4af7-80ea-68de2f2c66e8 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:01.173651 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86dab4-eca8-4af7-80ea-68de2f2c66e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:01.174341 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86dab4-eca8-4af7-80ea-68de2f2c66e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:01.175957 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86dab4-eca8-4af7-80ea-68de2f2c66e8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:03:01.176848 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86dab4-eca8-4af7-80ea-68de2f2c66e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:01.179650 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86dab4-eca8-4af7-80ea-68de2f2c66e8 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:01.196578 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 510/1028] 10.222.0.60 () {56 vars in 1115 bytes} [Wed Jul 13 23:03:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:01.203506 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df883c-49f8-4922-84a3-3bd6dbe88a13 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:01.351287 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df883c-49f8-4922-84a3-3bd6dbe88a13 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:01.398472 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae97f42-87db-4947-abaa-b39277f7cc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:01.399276 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae97f42-87db-4947-abaa-b39277f7cc51 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:01.399582 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae97f42-87db-4947-abaa-b39277f7cc51 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:01.399882 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae97f42-87db-4947-abaa-b39277f7cc51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:01.412861 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae97f42-87db-4947-abaa-b39277f7cc51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:01.414350 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae97f42-87db-4947-abaa-b39277f7cc51 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:01.434618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-eae97f42-87db-4947-abaa-b39277f7cc51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:03:01.434618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:03:01.434618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:03:01.434618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:03:01.434618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:03:01.434618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:03:01.434618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:03:01.434618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:03:01.434618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:03:01.434618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:03:01.434618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:03:01.434618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:03:01.434618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:03:01.434618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:03:01.434618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:03:01.436453 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:03:01.436453 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:03:01.436453 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:03:01.436453 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:03:01.436453 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:03:01.436453 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:03:01.436453 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:03:01.436453 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:03:01.436453 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:03:01.436453 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:03:01.436453 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:03:01.436453 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:03:01.436453 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:03:01.436453 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:03:01.436453 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:03:01.436453 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:03:01.436453 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:03:01.437691 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:03:01.437691 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:03:01.437691 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:03:01.437691 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:03:01.437691 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 519/1029] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:03:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:03:01.447242 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86dab4-eca8-4af7-80ea-68de2f2c66e8 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:01.636764 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86dab4-eca8-4af7-80ea-68de2f2c66e8 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:01.703420 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b1162-d5a1-493e-b428-790a9cd2101f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:01.704475 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b1162-d5a1-493e-b428-790a9cd2101f None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:01.704968 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b1162-d5a1-493e-b428-790a9cd2101f None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:01.705453 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b1162-d5a1-493e-b428-790a9cd2101f None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:01.706892 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36b1162-d5a1-493e-b428-790a9cd2101f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:03:01.707438 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36b1162-d5a1-493e-b428-790a9cd2101f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:01.710216 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36b1162-d5a1-493e-b428-790a9cd2101f None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:01.735417 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 511/1030] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:03:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:01.745498 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae97f42-87db-4947-abaa-b39277f7cc51 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:01.750757 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:01.939713 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae97f42-87db-4947-abaa-b39277f7cc51 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:02.003170 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec5a79-edbc-4ea0-9f76-aa30c66b3051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:02.004494 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec5a79-edbc-4ea0-9f76-aa30c66b3051 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:02.004992 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec5a79-edbc-4ea0-9f76-aa30c66b3051 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:02.005482 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec5a79-edbc-4ea0-9f76-aa30c66b3051 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:02.020734 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec5a79-edbc-4ea0-9f76-aa30c66b3051 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:02.023483 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec5a79-edbc-4ea0-9f76-aa30c66b3051 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:02.052141 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-90ec5a79-edbc-4ea0-9f76-aa30c66b3051 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 23:03:02.052141 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:03:02.052141 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:03:02.052141 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:03:02.052141 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:03:02.052141 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:03:02.052141 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:03:02.052141 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:03:02.052141 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:03:02.052141 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:03:02.052141 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:03:02.052141 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:03:02.052141 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 23:03:02.052141 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 23:03:02.052141 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 23:03:02.052141 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 23:03:02.053968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:03:02.053968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:03:02.053968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 23:03:02.053968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:03:02.053968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 23:03:02.053968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:03:02.053968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:03:02.053968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:03:02.053968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:03:02.053968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:03:02.053968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:03:02.053968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:03:02.053968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:03:02.053968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:03:02.053968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:03:02.053968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:03:02.053968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:03:02.053968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:03:02.056455 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:03:02.056455 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:03:02.056455 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 23:03:02.056455 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 23:03:02.056455 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 23:03:02.056455 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 23:03:02.056455 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 23:03:02.056455 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 23:03:02.056455 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 23:03:02.056455 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:03:02.058088 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 520/1031] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Jul 13 23:03:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:03:02.064105 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b1162-d5a1-493e-b428-790a9cd2101f None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:02.128787 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:02.254900 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b1162-d5a1-493e-b428-790a9cd2101f None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:02.312911 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3756cd9-e430-4887-8c03-acf26b0fe66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:02.314054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3756cd9-e430-4887-8c03-acf26b0fe66e None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:02.314543 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3756cd9-e430-4887-8c03-acf26b0fe66e None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:02.315021 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3756cd9-e430-4887-8c03-acf26b0fe66e None admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:02.316684 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3756cd9-e430-4887-8c03-acf26b0fe66e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:03:02.317763 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3756cd9-e430-4887-8c03-acf26b0fe66e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:02.320760 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3756cd9-e430-4887-8c03-acf26b0fe66e None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:02.359403 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 512/1032] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Jul 13 23:03:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:02.363956 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec5a79-edbc-4ea0-9f76-aa30c66b3051 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:02.475749 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:02.545330 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec5a79-edbc-4ea0-9f76-aa30c66b3051 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:02.604040 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b80807e-2cb5-4360-9a52-c141d3112c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:02.605136 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b80807e-2cb5-4360-9a52-c141d3112c18 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:02.605645 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b80807e-2cb5-4360-9a52-c141d3112c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:02.606143 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b80807e-2cb5-4360-9a52-c141d3112c18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:02.702404 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b80807e-2cb5-4360-9a52-c141d3112c18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:02.704222 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b80807e-2cb5-4360-9a52-c141d3112c18 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:02.831120 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:02.851167 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f4dad-ea4a-40f2-b439-1ba35ceaa6f0 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:02.852021 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f4dad-ea4a-40f2-b439-1ba35ceaa6f0 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:02.852021 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f4dad-ea4a-40f2-b439-1ba35ceaa6f0 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:02.853433 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 513/1033] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:03:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:03:02.859296 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866bdc68-36be-4dd4-adb9-17c72d96e076 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:02.859643 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866bdc68-36be-4dd4-adb9-17c72d96e076 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:02.859962 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866bdc68-36be-4dd4-adb9-17c72d96e076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:02.880510 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-866bdc68-36be-4dd4-adb9-17c72d96e076 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:03:02.881853 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-866bdc68-36be-4dd4-adb9-17c72d96e076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:03:02.904191 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 521/1034] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:03:02 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 23:03:02.910434 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b80807e-2cb5-4360-9a52-c141d3112c18 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:02.974465 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 514/1035] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:03:02 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 23:03:02.997785 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866bdc68-36be-4dd4-adb9-17c72d96e076 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:03.028041 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:03.069753 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b80807e-2cb5-4360-9a52-c141d3112c18 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:03.105527 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c8adf0-6aa5-4062-85e9-12abb55b0dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:03.106200 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c8adf0-6aa5-4062-85e9-12abb55b0dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:03.106482 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c8adf0-6aa5-4062-85e9-12abb55b0dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:03.106749 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c8adf0-6aa5-4062-85e9-12abb55b0dd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:03.119836 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c8adf0-6aa5-4062-85e9-12abb55b0dd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:03.121312 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c8adf0-6aa5-4062-85e9-12abb55b0dd1 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:03.125578 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866bdc68-36be-4dd4-adb9-17c72d96e076 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:03.140080 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-b5c8adf0-6aa5-4062-85e9-12abb55b0dd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:03:03.140080 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:03:03.140080 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:03:03.140080 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:03:03.140080 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:03:03.140080 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:03:03.140080 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:03:03.140080 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:03:03.140080 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:03:03.140080 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:03:03.140080 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:03:03.140080 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:03:03.140080 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:03:03.140080 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:03:03.140080 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:03:03.141254 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:03:03.141254 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:03:03.141254 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:03:03.141254 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:03:03.141254 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:03:03.141254 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:03:03.141254 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:03:03.141254 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:03:03.141254 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:03:03.141254 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:03:03.141254 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:03:03.141254 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:03:03.141254 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:03:03.141254 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:03:03.141254 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:03:03.141254 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:03:03.141254 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:03:03.142335 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:03:03.142335 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:03:03.142335 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:03:03.142335 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:03:03.142335 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 522/1036] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:03:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:03:03.142715 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76bbc5a-7e95-4b6c-bd22-059985034f45 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:03.142987 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76bbc5a-7e95-4b6c-bd22-059985034f45 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:03.143325 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76bbc5a-7e95-4b6c-bd22-059985034f45 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:03.144784 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 523/1037] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:03:03.148086 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76bbc5a-7e95-4b6c-bd22-059985034f45 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:03.165663 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:03.179861 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b710ea-1fe5-4b4c-9a5b-47aaaba2388f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:03.181470 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b710ea-1fe5-4b4c-9a5b-47aaaba2388f None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:03.181925 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b710ea-1fe5-4b4c-9a5b-47aaaba2388f None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:03.182526 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b710ea-1fe5-4b4c-9a5b-47aaaba2388f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:03.202666 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b710ea-1fe5-4b4c-9a5b-47aaaba2388f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:03:03.217996 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-30b710ea-1fe5-4b4c-9a5b-47aaaba2388f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:03:03.217996 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:03:03.217996 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:03:03.217996 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:03:03.217996 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:03:03.217996 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:03:03.217996 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:03:03.217996 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:03:03.217996 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:03:03.217996 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:03:03.217996 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:03:03.217996 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:03:03.217996 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:03:03.217996 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:03:03.217996 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:03:03.217996 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:03:03.219034 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:03:03.219034 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:03:03.219034 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:03:03.219034 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:03:03.219034 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:03:03.219034 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:03:03.219034 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:03:03.219034 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:03:03.219034 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:03:03.219034 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:03:03.219034 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:03:03.219034 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:03:03.219034 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:03:03.219034 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:03:03.219034 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:03:03.219034 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:03:03.219910 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:03:03.219910 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:03:03.219910 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:03:03.219910 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:03:03.219910 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:03:03.219910 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:03:03.219910 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:03:03.219910 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:03:03.219910 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:03:03.219910 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:03:03.219910 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:03:03.219910 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:03:03.220679 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 515/1038] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jul 13 23:03:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:03:03.221943 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d701ace4-ca3c-473c-aec7-f475386af7ff None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:03.222436 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d701ace4-ca3c-473c-aec7-f475386af7ff None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:03.223489 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d701ace4-ca3c-473c-aec7-f475386af7ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:03.250225 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-d701ace4-ca3c-473c-aec7-f475386af7ff None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:03:03.252072 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-d701ace4-ca3c-473c-aec7-f475386af7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:03:03.261194 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76bbc5a-7e95-4b6c-bd22-059985034f45 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:03.301110 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d6156-cad1-4ba2-a8f1-bec2e3f5ba7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:03.301822 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d6156-cad1-4ba2-a8f1-bec2e3f5ba7c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:03.302104 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d6156-cad1-4ba2-a8f1-bec2e3f5ba7c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:03.302377 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d6156-cad1-4ba2-a8f1-bec2e3f5ba7c None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:03.303184 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863d6156-cad1-4ba2-a8f1-bec2e3f5ba7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:03:03.303426 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863d6156-cad1-4ba2-a8f1-bec2e3f5ba7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:03.304872 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863d6156-cad1-4ba2-a8f1-bec2e3f5ba7c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:03.317631 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 524/1039] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:03:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:03:03.318492 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d6156-cad1-4ba2-a8f1-bec2e3f5ba7c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:03.368245 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 516/1040] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:03:03 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:03:03.369170 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d701ace4-ca3c-473c-aec7-f475386af7ff None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:03.390362 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:03.412256 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:03.445480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d6156-cad1-4ba2-a8f1-bec2e3f5ba7c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:03.488166 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8039a6-50d0-43a2-abf0-9abcfe3da976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:03.489530 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8039a6-50d0-43a2-abf0-9abcfe3da976 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:03.489873 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8039a6-50d0-43a2-abf0-9abcfe3da976 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:03.490280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8039a6-50d0-43a2-abf0-9abcfe3da976 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:03.491216 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8039a6-50d0-43a2-abf0-9abcfe3da976 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:03:03.491708 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8039a6-50d0-43a2-abf0-9abcfe3da976 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:03.493285 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8039a6-50d0-43a2-abf0-9abcfe3da976 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:03.493432 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d701ace4-ca3c-473c-aec7-f475386af7ff None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:03.503868 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 525/1041] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:03:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:03.505271 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8039a6-50d0-43a2-abf0-9abcfe3da976 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:03.538286 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7580a0d1-8f25-4aac-a8a2-6d97738bb33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:03.539215 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580a0d1-8f25-4aac-a8a2-6d97738bb33a None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:03.539496 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580a0d1-8f25-4aac-a8a2-6d97738bb33a None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:03.539802 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580a0d1-8f25-4aac-a8a2-6d97738bb33a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:03.547698 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7580a0d1-8f25-4aac-a8a2-6d97738bb33a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:03.549224 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7580a0d1-8f25-4aac-a8a2-6d97738bb33a None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:03.564720 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-7580a0d1-8f25-4aac-a8a2-6d97738bb33a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:03:03.564720 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:03:03.564720 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:03:03.564720 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:03:03.564720 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:03:03.564720 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:03:03.564720 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:03:03.564720 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:03:03.564720 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:03:03.564720 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:03:03.564720 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:03:03.564720 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:03:03.564720 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 23:03:03.564720 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 23:03:03.564720 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 23:03:03.566528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 23:03:03.566528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:03:03.566528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:03:03.566528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:03:03.566528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:03:03.566528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:03:03.566528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:03:03.566528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:03:03.566528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:03:03.566528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:03:03.566528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:03:03.566528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:03:03.566528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:03:03.566528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:03:03.566528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:03:03.566528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:03:03.566528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:03:03.568745 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 23:03:03.568745 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 23:03:03.568745 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 23:03:03.568745 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 23:03:03.568745 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 23:03:03.568745 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 23:03:03.568745 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:03:03.568745 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:03:03.568745 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 517/1042] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:03:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:03:03.568745 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7580a0d1-8f25-4aac-a8a2-6d97738bb33a None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:03.627172 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8039a6-50d0-43a2-abf0-9abcfe3da976 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:03.664852 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:03.666622 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072cb7a5-f924-4ad4-902b-db051683ec46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:03.667444 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072cb7a5-f924-4ad4-902b-db051683ec46 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:03.667755 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072cb7a5-f924-4ad4-902b-db051683ec46 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:03.668034 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072cb7a5-f924-4ad4-902b-db051683ec46 None admin] PATH_INFO: `/v3/services/image` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:03.668701 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072cb7a5-f924-4ad4-902b-db051683ec46 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:03.670107 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072cb7a5-f924-4ad4-902b-db051683ec46 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:03.673213 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:03.684358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-072cb7a5-f924-4ad4-902b-db051683ec46 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 23:03:03.684358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:03:03.684358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 23:03:03.684358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 23:03:03.684358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 23:03:03.684358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 23:03:03.684358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 23:03:03.684358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:03:03.684358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 23:03:03.684358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:03:03.684358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 23:03:03.684358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:03:03.684358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:03:03.684358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:03:03.684358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:03:03.684358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:03:03.684358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:03:03.685707 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:03:03.685707 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:03:03.685707 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:03:03.685707 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:03:03.685707 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:03:03.685707 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:03:03.685707 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 23:03:03.685707 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 23:03:03.685707 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 23:03:03.685707 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 23:03:03.685707 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:03:03.685707 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:03:03.685707 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:03:03.685707 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:03:03.685707 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:03:03.685707 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:03:03.686770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:03:03.686770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:03:03.686770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:03:03.686770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:03:03.686770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:03:03.686770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:03:03.686770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:03:03.686770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:03:03.686770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:03:03.686770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:03:03.686770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 23:03:03.686770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:03:03.686770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 23:03:03.686770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:03:03.689251 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 526/1043] 10.222.0.60 () {56 vars in 1105 bytes} [Wed Jul 13 23:03:03 2022] GET /identity/v3/services/image => generated 86 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:03:03.690092 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072cb7a5-f924-4ad4-902b-db051683ec46 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:03.718790 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7580a0d1-8f25-4aac-a8a2-6d97738bb33a None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:03.756690 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b830fdf4-b898-4860-956b-fe0b8a9b12e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:03.757580 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830fdf4-b898-4860-956b-fe0b8a9b12e0 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:03.757857 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830fdf4-b898-4860-956b-fe0b8a9b12e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:03.758137 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830fdf4-b898-4860-956b-fe0b8a9b12e0 None admin] PATH_INFO: `/v3/users/fa93f188af7945a08840d5dda10d52a8` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:03.774503 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b830fdf4-b898-4860-956b-fe0b8a9b12e0 None admin] RBAC: Authorizing `identity:get_user(user_id=fa93f188af7945a08840d5dda10d52a8)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:03.776235 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b830fdf4-b898-4860-956b-fe0b8a9b12e0 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:03.778414 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 518/1044] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:03:03 2022] GET /identity/v3/users/fa93f188af7945a08840d5dda10d52a8 => generated 285 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:03.779818 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b830fdf4-b898-4860-956b-fe0b8a9b12e0 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:03.835568 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072cb7a5-f924-4ad4-902b-db051683ec46 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:03.877623 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929e62e-af30-44e2-8bb3-931ed87d209c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:03.878359 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929e62e-af30-44e2-8bb3-931ed87d209c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:03.878749 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929e62e-af30-44e2-8bb3-931ed87d209c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:03.879125 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929e62e-af30-44e2-8bb3-931ed87d209c None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:03.880109 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0929e62e-af30-44e2-8bb3-931ed87d209c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:03:03.880634 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0929e62e-af30-44e2-8bb3-931ed87d209c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:03.882467 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0929e62e-af30-44e2-8bb3-931ed87d209c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:03.907136 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 527/1045] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jul 13 23:03:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:03.907136 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929e62e-af30-44e2-8bb3-931ed87d209c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:03.917081 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b830fdf4-b898-4860-956b-fe0b8a9b12e0 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:03.955019 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:03.963671 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754e078-c20e-42dd-a6f7-2dcf9a787a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:03.964439 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754e078-c20e-42dd-a6f7-2dcf9a787a1d None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:03.964869 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754e078-c20e-42dd-a6f7-2dcf9a787a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:03.965230 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754e078-c20e-42dd-a6f7-2dcf9a787a1d None admin] PATH_INFO: `/v3/services` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:03.966115 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7754e078-c20e-42dd-a6f7-2dcf9a787a1d None admin] RBAC: Adding query filter params (name=image) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:03:03.966519 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7754e078-c20e-42dd-a6f7-2dcf9a787a1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:03.968103 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7754e078-c20e-42dd-a6f7-2dcf9a787a1d None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:03.974391 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 519/1046] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:03:03 2022] GET /identity/v3/services?name=image => generated 122 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:03.975356 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754e078-c20e-42dd-a6f7-2dcf9a787a1d None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:03.984276 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:04.039102 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929e62e-af30-44e2-8bb3-931ed87d209c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:04.088672 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac69a520-4b90-48b3-b788-b2431d491653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:04.089443 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac69a520-4b90-48b3-b788-b2431d491653 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:04.089790 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac69a520-4b90-48b3-b788-b2431d491653 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:04.090129 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac69a520-4b90-48b3-b788-b2431d491653 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:04.112253 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754e078-c20e-42dd-a6f7-2dcf9a787a1d None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:04.151002 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e7841e-312e-4bd3-a198-95b62924b2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:04.151588 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7841e-312e-4bd3-a198-95b62924b2c7 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:04.151873 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7841e-312e-4bd3-a198-95b62924b2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:04.152152 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7841e-312e-4bd3-a198-95b62924b2c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:04.153016 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e7841e-312e-4bd3-a198-95b62924b2c7 None admin] RBAC: Adding query filter params (role.id=f7411bbcd41a4f959f76b130485bfb89, scope.project.id=ab0a5581e4e444c293815e1304e20c05, user.id=4789384d158b4995bd30e1e37e30f5f1) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:03:04.153330 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e7841e-312e-4bd3-a198-95b62924b2c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:04.154827 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e7841e-312e-4bd3-a198-95b62924b2c7 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:04.161889 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 520/1047] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:03:03 2022] GET /identity/v3/role_assignments?user.id=4789384d158b4995bd30e1e37e30f5f1&scope.project.id=ab0a5581e4e444c293815e1304e20c05&role.id=f7411bbcd41a4f959f76b130485bfb89 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:04.162738 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e7841e-312e-4bd3-a198-95b62924b2c7 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:04.190738 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac69a520-4b90-48b3-b788-b2431d491653 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:04.193664 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac69a520-4b90-48b3-b788-b2431d491653 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:04.235715 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:04.290868 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e7841e-312e-4bd3-a198-95b62924b2c7 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:04.305267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:04.347292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c749a-eb2a-45ad-b55b-21857ab482a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:04.348093 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c749a-eb2a-45ad-b55b-21857ab482a9 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:04.349339 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c749a-eb2a-45ad-b55b-21857ab482a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:04.349625 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c749a-eb2a-45ad-b55b-21857ab482a9 None admin] PATH_INFO: `/v3/services` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:04.350368 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c749a-eb2a-45ad-b55b-21857ab482a9 None admin] RBAC: Adding query filter params (type=image) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:03:04.350675 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c749a-eb2a-45ad-b55b-21857ab482a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:04.356811 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c749a-eb2a-45ad-b55b-21857ab482a9 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:04.369111 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 521/1048] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:03:04 2022] GET /identity/v3/services?type=image => generated 350 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:04.369469 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c749a-eb2a-45ad-b55b-21857ab482a9 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:04.388180 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 528/1049] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:03:03 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 23:03:04.398142 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac69a520-4b90-48b3-b788-b2431d491653 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:04.493533 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c749a-eb2a-45ad-b55b-21857ab482a9 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:04.536816 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:04.556992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491b6ca-5666-418f-82a1-c7491506ba22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:04.558105 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491b6ca-5666-418f-82a1-c7491506ba22 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:04.558595 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491b6ca-5666-418f-82a1-c7491506ba22 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:04.559077 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491b6ca-5666-418f-82a1-c7491506ba22 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:04.560610 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6491b6ca-5666-418f-82a1-c7491506ba22 None admin] RBAC: Adding query filter params (interface=public, service_id=a278a85046044b0b9828bafc3e5ae568, region_id=RegionOne) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:03:04.561154 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6491b6ca-5666-418f-82a1-c7491506ba22 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:04.563713 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6491b6ca-5666-418f-82a1-c7491506ba22 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:04.573820 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac69a520-4b90-48b3-b788-b2431d491653 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:04.574026 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 522/1050] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Jul 13 23:03:04 2022] GET /identity/v3/endpoints?service_id=a278a85046044b0b9828bafc3e5ae568&interface=public®ion_id=RegionOne => generated 194 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:03:04.583891 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491b6ca-5666-418f-82a1-c7491506ba22 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:04.635511 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f3fef1-1beb-4c30-8a48-5b17784829b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:04.637183 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3fef1-1beb-4c30-8a48-5b17784829b6 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:04.637692 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3fef1-1beb-4c30-8a48-5b17784829b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:04.638199 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3fef1-1beb-4c30-8a48-5b17784829b6 None admin] PATH_INFO: `/v3/projects/cfff177a8165466dac56ca0ec7923f3e` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:04.640249 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f3fef1-1beb-4c30-8a48-5b17784829b6 None admin] RBAC: Authorizing `identity:get_project(project_id=cfff177a8165466dac56ca0ec7923f3e)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:04.642619 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f3fef1-1beb-4c30-8a48-5b17784829b6 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:04.651482 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 529/1051] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:03:04 2022] GET /identity/v3/projects/cfff177a8165466dac56ca0ec7923f3e => generated 311 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:04.653520 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f3fef1-1beb-4c30-8a48-5b17784829b6 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:04.702234 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:04.751281 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491b6ca-5666-418f-82a1-c7491506ba22 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:04.812770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f3fef1-1beb-4c30-8a48-5b17784829b6 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:04.814163 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c455e31d-a6ec-4373-af2d-4fd9bb6e0a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:04.815015 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455e31d-a6ec-4373-af2d-4fd9bb6e0a63 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:04.815472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455e31d-a6ec-4373-af2d-4fd9bb6e0a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:04.815926 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455e31d-a6ec-4373-af2d-4fd9bb6e0a63 None admin] PATH_INFO: `/v3/services` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:04.817208 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c455e31d-a6ec-4373-af2d-4fd9bb6e0a63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:04.818948 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c455e31d-a6ec-4373-af2d-4fd9bb6e0a63 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:04.827127 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 523/1052] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Jul 13 23:03:04 2022] GET /identity/v3/services => generated 1957 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:03:04.890780 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1b336-751c-481e-83f8-94b81267d41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:04.891976 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1b336-751c-481e-83f8-94b81267d41b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:04.892464 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1b336-751c-481e-83f8-94b81267d41b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:04.892975 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1b336-751c-481e-83f8-94b81267d41b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:04.894696 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa1b336-751c-481e-83f8-94b81267d41b None admin] RBAC: Adding query filter params (role.id=8cfd0b6e71e14e0983c3eb4d7478e027, scope.project.id=cfff177a8165466dac56ca0ec7923f3e, user.id=fa93f188af7945a08840d5dda10d52a8) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:03:04.895319 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa1b336-751c-481e-83f8-94b81267d41b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:04.898104 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa1b336-751c-481e-83f8-94b81267d41b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:04.915894 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 530/1053] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:03:04 2022] GET /identity/v3/role_assignments?user.id=fa93f188af7945a08840d5dda10d52a8&scope.project.id=cfff177a8165466dac56ca0ec7923f3e&role.id=8cfd0b6e71e14e0983c3eb4d7478e027 => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:03:06.317860 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320df18-78fa-44e2-bb4a-a0d204e83f92 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:06.318433 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320df18-78fa-44e2-bb4a-a0d204e83f92 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:06.318744 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320df18-78fa-44e2-bb4a-a0d204e83f92 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:06.321424 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 524/1054] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:03:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:03:06.328150 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36cfa49-c558-4f9b-9a25-314cb3490217 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:06.328687 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36cfa49-c558-4f9b-9a25-314cb3490217 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:06.329229 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36cfa49-c558-4f9b-9a25-314cb3490217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:06.352380 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-a36cfa49-c558-4f9b-9a25-314cb3490217 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:03:06.353612 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a36cfa49-c558-4f9b-9a25-314cb3490217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:03:06.440803 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 531/1055] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:03:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:03:06.465063 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7320df18-78fa-44e2-bb4a-a0d204e83f92 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:06.557845 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:06.625147 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7320df18-78fa-44e2-bb4a-a0d204e83f92 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:06.663905 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644cb4b-9f0d-4e5c-b79d-945fa252b635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:06.664734 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644cb4b-9f0d-4e5c-b79d-945fa252b635 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:06.665011 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644cb4b-9f0d-4e5c-b79d-945fa252b635 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:06.665275 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e644cb4b-9f0d-4e5c-b79d-945fa252b635 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:06.678153 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e644cb4b-9f0d-4e5c-b79d-945fa252b635 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:03:06.686240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-e644cb4b-9f0d-4e5c-b79d-945fa252b635 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 23:03:06.686240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:03:06.686240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:03:06.686240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:03:06.686240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:03:06.686240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:03:06.686240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:03:06.686240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:03:06.686240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:03:06.686240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:03:06.686240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:03:06.686240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:03:06.686240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:03:06.686240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:03:06.686240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:03:06.686240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:03:06.688054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:03:06.688054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:03:06.688054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:03:06.688054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:03:06.688054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:03:06.688054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:03:06.688054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:03:06.688054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:03:06.688054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:03:06.688054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:03:06.688054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:03:06.688054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:03:06.688054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:03:06.688054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:03:06.688054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:03:06.688054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:03:06.689106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:03:06.689106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:03:06.689106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:03:06.689106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:03:06.689106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:03:06.689106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:03:06.689106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:03:06.689106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:03:06.689106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:03:06.689106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:03:06.689106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 23:03:06.689106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:03:06.689106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 525/1056] 10.222.0.60 () {56 vars in 1120 bytes} [Wed Jul 13 23:03:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:03:06.694360 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36cfa49-c558-4f9b-9a25-314cb3490217 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:06.720130 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:06.808862 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36cfa49-c558-4f9b-9a25-314cb3490217 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:06.856097 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e2c14c-d1c7-46dc-a31f-98f9740ef4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:06.856771 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e2c14c-d1c7-46dc-a31f-98f9740ef4e3 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:06.857052 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e2c14c-d1c7-46dc-a31f-98f9740ef4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:06.857334 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e2c14c-d1c7-46dc-a31f-98f9740ef4e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:06.858083 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e2c14c-d1c7-46dc-a31f-98f9740ef4e3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:03:06.858387 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e2c14c-d1c7-46dc-a31f-98f9740ef4e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:06.859779 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e2c14c-d1c7-46dc-a31f-98f9740ef4e3 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:06.873887 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 532/1057] 10.222.0.60 () {56 vars in 1115 bytes} [Wed Jul 13 23:03:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:06.879031 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644cb4b-9f0d-4e5c-b79d-945fa252b635 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:06.989616 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e644cb4b-9f0d-4e5c-b79d-945fa252b635 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:07.008843 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:07.044546 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5925669-1cdf-4fd0-ad5f-3652140180d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:07.045691 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5925669-1cdf-4fd0-ad5f-3652140180d6 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:07.046352 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5925669-1cdf-4fd0-ad5f-3652140180d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:07.046806 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5925669-1cdf-4fd0-ad5f-3652140180d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:07.061270 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5925669-1cdf-4fd0-ad5f-3652140180d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:07.063113 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5925669-1cdf-4fd0-ad5f-3652140180d6 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:07.092654 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-b5925669-1cdf-4fd0-ad5f-3652140180d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:03:07.092654 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:03:07.092654 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:03:07.092654 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:03:07.092654 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:03:07.092654 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:03:07.092654 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:03:07.092654 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:03:07.092654 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:03:07.092654 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:03:07.092654 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:03:07.092654 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:03:07.092654 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:03:07.092654 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:03:07.092654 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:03:07.094554 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:03:07.094554 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:03:07.094554 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:03:07.094554 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:03:07.094554 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:03:07.094554 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:03:07.094554 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:03:07.094554 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:03:07.094554 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:03:07.094554 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:03:07.094554 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:03:07.094554 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:03:07.094554 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:03:07.094554 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:03:07.094554 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:03:07.094554 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:03:07.094554 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:03:07.096460 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:03:07.096460 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:03:07.096460 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:03:07.096460 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:03:07.099765 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 526/1058] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:03:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:03:07.104549 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e2c14c-d1c7-46dc-a31f-98f9740ef4e3 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:07.249347 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:07.268658 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e2c14c-d1c7-46dc-a31f-98f9740ef4e3 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:07.309082 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee78ed-9414-4662-ba2b-d28e6bcd5297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:07.309820 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee78ed-9414-4662-ba2b-d28e6bcd5297 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:07.310095 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee78ed-9414-4662-ba2b-d28e6bcd5297 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:07.310367 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee78ed-9414-4662-ba2b-d28e6bcd5297 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:07.311149 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee78ed-9414-4662-ba2b-d28e6bcd5297 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:03:07.311454 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee78ed-9414-4662-ba2b-d28e6bcd5297 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:07.312867 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee78ed-9414-4662-ba2b-d28e6bcd5297 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:07.332647 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 533/1059] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:03:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:07.335074 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5925669-1cdf-4fd0-ad5f-3652140180d6 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:07.484995 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5925669-1cdf-4fd0-ad5f-3652140180d6 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:07.540377 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff67dac-5301-40c5-8773-b61e6d18632d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:07.541078 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff67dac-5301-40c5-8773-b61e6d18632d None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:07.541359 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff67dac-5301-40c5-8773-b61e6d18632d None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:07.541615 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff67dac-5301-40c5-8773-b61e6d18632d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:07.550860 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff67dac-5301-40c5-8773-b61e6d18632d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:07.552947 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff67dac-5301-40c5-8773-b61e6d18632d None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:07.568291 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-aff67dac-5301-40c5-8773-b61e6d18632d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 23:03:07.568291 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:03:07.568291 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:03:07.568291 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:03:07.568291 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:03:07.568291 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:03:07.568291 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:03:07.568291 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:03:07.568291 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:03:07.568291 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:03:07.568291 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:03:07.568291 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:03:07.568291 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 23:03:07.568291 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 23:03:07.568291 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 23:03:07.568291 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 23:03:07.570453 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:03:07.570453 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:03:07.570453 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 23:03:07.570453 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:03:07.570453 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 23:03:07.570453 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:03:07.570453 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:03:07.570453 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:03:07.570453 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:03:07.570453 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:03:07.570453 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:03:07.570453 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:03:07.570453 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:03:07.570453 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:03:07.570453 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:03:07.570453 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:03:07.570453 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:03:07.570453 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:03:07.572985 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:03:07.572985 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:03:07.572985 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 23:03:07.572985 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 23:03:07.572985 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 23:03:07.572985 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 23:03:07.572985 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 23:03:07.572985 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 23:03:07.572985 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 23:03:07.572985 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:03:07.579571 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 527/1060] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Jul 13 23:03:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:03:07.579571 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff67dac-5301-40c5-8773-b61e6d18632d None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:07.579913 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a9451-08f7-4c84-848e-ee038a49ebbf None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:07.579913 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a9451-08f7-4c84-848e-ee038a49ebbf None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:07.579913 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a9451-08f7-4c84-848e-ee038a49ebbf None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:07.580862 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 534/1061] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:03:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:03:07.586688 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83dc2ca-2d5d-46c4-abab-3d9fe6814f0c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:07.586995 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83dc2ca-2d5d-46c4-abab-3d9fe6814f0c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:07.587303 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83dc2ca-2d5d-46c4-abab-3d9fe6814f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:07.605915 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c83dc2ca-2d5d-46c4-abab-3d9fe6814f0c None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:03:07.606809 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:07.607145 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c83dc2ca-2d5d-46c4-abab-3d9fe6814f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:03:07.713356 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 535/1062] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:03:07 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 23:03:07.714888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf1f1ba-c6ff-4c6a-ada8-317939b7f459 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:07.715181 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf1f1ba-c6ff-4c6a-ada8-317939b7f459 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:07.715458 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf1f1ba-c6ff-4c6a-ada8-317939b7f459 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:07.720296 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 536/1063] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:03:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:03:07.725855 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cad9cc-7c2a-49c4-9608-bff7c3813048 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:07.726731 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cad9cc-7c2a-49c4-9608-bff7c3813048 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:07.726731 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cad9cc-7c2a-49c4-9608-bff7c3813048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:07.742766 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff67dac-5301-40c5-8773-b61e6d18632d None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:07.747816 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-90cad9cc-7c2a-49c4-9608-bff7c3813048 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:03:07.749096 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-90cad9cc-7c2a-49c4-9608-bff7c3813048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:03:07.801568 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f6d6d2-6d04-4779-a88f-4779f5aacb38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:07.802576 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6d6d2-6d04-4779-a88f-4779f5aacb38 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:07.803076 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6d6d2-6d04-4779-a88f-4779f5aacb38 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:07.803566 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6d6d2-6d04-4779-a88f-4779f5aacb38 None admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:07.804965 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f6d6d2-6d04-4779-a88f-4779f5aacb38 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:03:07.805531 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f6d6d2-6d04-4779-a88f-4779f5aacb38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:07.808298 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f6d6d2-6d04-4779-a88f-4779f5aacb38 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:07.828726 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 537/1064] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:03:07 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 23:03:07.829611 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cad9cc-7c2a-49c4-9608-bff7c3813048 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:07.835474 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 528/1065] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Jul 13 23:03:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:07.841340 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f6d6d2-6d04-4779-a88f-4779f5aacb38 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:07.841985 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:07.968478 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:07.983704 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cad9cc-7c2a-49c4-9608-bff7c3813048 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:07.990469 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f6d6d2-6d04-4779-a88f-4779f5aacb38 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:08.022292 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d13752-b675-4be0-ac34-bdee44ca18e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:08.023092 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d13752-b675-4be0-ac34-bdee44ca18e2 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:08.023713 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d13752-b675-4be0-ac34-bdee44ca18e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:08.023713 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d13752-b675-4be0-ac34-bdee44ca18e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:08.035032 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d13752-b675-4be0-ac34-bdee44ca18e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:03:08.035338 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1677a4d4-4a5c-482e-881f-c71bbc335386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:08.036802 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677a4d4-4a5c-482e-881f-c71bbc335386 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:08.037083 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677a4d4-4a5c-482e-881f-c71bbc335386 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:08.037361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677a4d4-4a5c-482e-881f-c71bbc335386 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:08.043483 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-e7d13752-b675-4be0-ac34-bdee44ca18e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:03:08.043483 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:03:08.043483 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:03:08.043483 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:03:08.043483 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:03:08.043483 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:03:08.043483 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:03:08.043483 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:03:08.043483 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:03:08.043483 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:03:08.043483 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:03:08.043483 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:03:08.043483 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:03:08.043483 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:03:08.043483 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:03:08.043483 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:03:08.045619 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:03:08.045619 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:03:08.045619 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:03:08.045619 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:03:08.045619 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:03:08.045619 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:03:08.045619 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:03:08.045619 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:03:08.045619 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:03:08.045619 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:03:08.045619 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:03:08.045619 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:03:08.045619 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:03:08.045619 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:03:08.045619 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:03:08.045619 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:03:08.049218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:03:08.049218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:03:08.049218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:03:08.049218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:03:08.049218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:03:08.049218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:03:08.049218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:03:08.049218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:03:08.049218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:03:08.049218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:03:08.049218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:03:08.049218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:03:08.049218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 538/1066] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jul 13 23:03:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:03:08.049218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d13752-b675-4be0-ac34-bdee44ca18e2 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:08.054780 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1677a4d4-4a5c-482e-881f-c71bbc335386 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:08.056456 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1677a4d4-4a5c-482e-881f-c71bbc335386 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:08.090338 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-1677a4d4-4a5c-482e-881f-c71bbc335386 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:03:08.090338 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:03:08.090338 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:03:08.090338 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:03:08.090338 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:03:08.090338 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:03:08.090338 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:03:08.090338 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:03:08.090338 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:03:08.090338 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:03:08.090338 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:03:08.090338 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:03:08.090338 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:03:08.090338 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:03:08.090338 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:03:08.091762 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:03:08.091762 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:03:08.091762 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:03:08.091762 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:03:08.091762 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:03:08.091762 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:03:08.091762 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:03:08.091762 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:03:08.091762 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:03:08.091762 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:03:08.091762 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:03:08.091762 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:03:08.091762 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:03:08.091762 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:03:08.091762 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:03:08.091762 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:03:08.091762 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:03:08.092661 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:03:08.092661 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:03:08.092661 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:03:08.092661 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:03:08.095730 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 529/1067] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:03:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:03:08.097193 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1677a4d4-4a5c-482e-881f-c71bbc335386 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:08.118931 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:08.167719 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d13752-b675-4be0-ac34-bdee44ca18e2 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:08.208447 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f964a-6ea3-4562-a83c-9d9de6e65c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:08.209263 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f964a-6ea3-4562-a83c-9d9de6e65c0f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:08.209541 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f964a-6ea3-4562-a83c-9d9de6e65c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:08.209812 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f964a-6ea3-4562-a83c-9d9de6e65c0f None admin] PATH_INFO: `/v3/services/image` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:08.210487 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60f964a-6ea3-4562-a83c-9d9de6e65c0f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:08.211944 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60f964a-6ea3-4562-a83c-9d9de6e65c0f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:08.224986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-b60f964a-6ea3-4562-a83c-9d9de6e65c0f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 23:03:08.224986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:03:08.224986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 23:03:08.224986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 23:03:08.224986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 23:03:08.224986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 23:03:08.224986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 23:03:08.224986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:03:08.224986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 23:03:08.224986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:03:08.224986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 23:03:08.224986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:03:08.224986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:03:08.224986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:03:08.224986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:03:08.224986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:03:08.224986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:03:08.226270 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:03:08.226270 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:03:08.226270 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:03:08.226270 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:03:08.226270 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:03:08.226270 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:03:08.226270 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 23:03:08.226270 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 23:03:08.226270 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 23:03:08.226270 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 23:03:08.226270 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:03:08.226270 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:03:08.226270 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:03:08.226270 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:03:08.226270 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:03:08.226270 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:03:08.227105 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:03:08.227105 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:03:08.227105 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:03:08.227105 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:03:08.227105 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:03:08.227105 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:03:08.227105 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:03:08.227105 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:03:08.227105 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:03:08.227105 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:03:08.227105 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 23:03:08.227105 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:03:08.227105 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 23:03:08.227105 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:03:08.227105 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 539/1068] 10.222.0.60 () {56 vars in 1105 bytes} [Wed Jul 13 23:03:08 2022] GET /identity/v3/services/image => generated 86 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:03:08.228077 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f964a-6ea3-4562-a83c-9d9de6e65c0f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:08.235279 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1677a4d4-4a5c-482e-881f-c71bbc335386 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:08.269293 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:08.276001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c323c-dca3-4a61-9935-e7b5fe719445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:08.276687 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c323c-dca3-4a61-9935-e7b5fe719445 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:08.276964 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c323c-dca3-4a61-9935-e7b5fe719445 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:08.277241 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c323c-dca3-4a61-9935-e7b5fe719445 None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:08.277996 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c323c-dca3-4a61-9935-e7b5fe719445 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:03:08.278303 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c323c-dca3-4a61-9935-e7b5fe719445 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:08.279728 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c323c-dca3-4a61-9935-e7b5fe719445 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:08.289372 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 530/1069] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:03:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:08.290244 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c323c-dca3-4a61-9935-e7b5fe719445 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:08.358393 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f964a-6ea3-4562-a83c-9d9de6e65c0f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:08.391038 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:08.402334 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018d40d-693f-4994-9285-16e115e1f4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:08.403015 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018d40d-693f-4994-9285-16e115e1f4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:08.403302 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018d40d-693f-4994-9285-16e115e1f4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:08.403620 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018d40d-693f-4994-9285-16e115e1f4f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:08.404368 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1018d40d-693f-4994-9285-16e115e1f4f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:03:08.404677 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1018d40d-693f-4994-9285-16e115e1f4f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:08.406094 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1018d40d-693f-4994-9285-16e115e1f4f2 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:08.419765 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c323c-dca3-4a61-9935-e7b5fe719445 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:08.420562 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 540/1070] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:03:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:08.420906 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018d40d-693f-4994-9285-16e115e1f4f2 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:08.466532 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68663b56-7459-419d-b734-ce711a58798a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:08.467227 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68663b56-7459-419d-b734-ce711a58798a None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:08.467516 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68663b56-7459-419d-b734-ce711a58798a None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:08.467825 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68663b56-7459-419d-b734-ce711a58798a None admin] PATH_INFO: `/v3/services` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:08.468552 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68663b56-7459-419d-b734-ce711a58798a None admin] RBAC: Adding query filter params (name=image) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:03:08.468853 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68663b56-7459-419d-b734-ce711a58798a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:08.470260 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68663b56-7459-419d-b734-ce711a58798a None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:08.476187 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 531/1071] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:03:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:08.477048 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68663b56-7459-419d-b734-ce711a58798a None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:08.541383 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018d40d-693f-4994-9285-16e115e1f4f2 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:08.544486 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:08.580089 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f60f1-e18d-4163-9932-847187168933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:08.580962 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f60f1-e18d-4163-9932-847187168933 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:08.581247 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f60f1-e18d-4163-9932-847187168933 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:08.581516 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f60f1-e18d-4163-9932-847187168933 None admin] PATH_INFO: `/v3/users/fa93f188af7945a08840d5dda10d52a8` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:08.582759 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0f60f1-e18d-4163-9932-847187168933 None admin] RBAC: Authorizing `identity:get_user(user_id=fa93f188af7945a08840d5dda10d52a8)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:08.584231 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0f60f1-e18d-4163-9932-847187168933 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:08.592184 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 541/1072] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:03:08 2022] GET /identity/v3/users/fa93f188af7945a08840d5dda10d52a8 => generated 285 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:08.592184 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f60f1-e18d-4163-9932-847187168933 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:08.595631 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68663b56-7459-419d-b734-ce711a58798a None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:08.638468 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e91260-c7e5-46b7-82d1-b8fa59b5a63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:08.639486 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e91260-c7e5-46b7-82d1-b8fa59b5a63c None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:08.639937 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e91260-c7e5-46b7-82d1-b8fa59b5a63c None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:08.640296 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e91260-c7e5-46b7-82d1-b8fa59b5a63c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:08.649617 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e91260-c7e5-46b7-82d1-b8fa59b5a63c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:08.651392 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e91260-c7e5-46b7-82d1-b8fa59b5a63c None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:08.653349 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:08.668378 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-47e91260-c7e5-46b7-82d1-b8fa59b5a63c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:03:08.668378 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:03:08.668378 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:03:08.668378 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:03:08.668378 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:03:08.668378 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:03:08.668378 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:03:08.668378 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:03:08.668378 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:03:08.668378 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:03:08.668378 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:03:08.668378 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:03:08.668378 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 23:03:08.668378 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 23:03:08.668378 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 23:03:08.670135 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 23:03:08.670135 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:03:08.670135 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:03:08.670135 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:03:08.670135 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:03:08.670135 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:03:08.670135 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:03:08.670135 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:03:08.670135 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:03:08.670135 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:03:08.670135 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:03:08.670135 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:03:08.670135 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:03:08.670135 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:03:08.670135 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:03:08.670135 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:03:08.670135 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:03:08.671106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 23:03:08.671106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 23:03:08.671106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 23:03:08.671106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 23:03:08.671106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 23:03:08.671106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 23:03:08.671106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:03:08.671106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:03:08.671106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 532/1073] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:03:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:03:08.671106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e91260-c7e5-46b7-82d1-b8fa59b5a63c None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:08.703036 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f60f1-e18d-4163-9932-847187168933 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:08.738040 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040cd814-517f-4057-992e-3b67dfce052f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:08.738743 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040cd814-517f-4057-992e-3b67dfce052f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:08.739021 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040cd814-517f-4057-992e-3b67dfce052f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:08.739305 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040cd814-517f-4057-992e-3b67dfce052f None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:08.740053 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040cd814-517f-4057-992e-3b67dfce052f None admin] RBAC: Adding query filter params (type=image) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:03:08.740411 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040cd814-517f-4057-992e-3b67dfce052f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:08.741798 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040cd814-517f-4057-992e-3b67dfce052f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:08.746864 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 542/1074] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:03:08 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:08.752074 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040cd814-517f-4057-992e-3b67dfce052f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:08.781522 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e91260-c7e5-46b7-82d1-b8fa59b5a63c None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:08.802236 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:08.823647 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdac09ac-159c-4294-a216-e4323acbfafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:08.824715 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac09ac-159c-4294-a216-e4323acbfafc None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:08.825093 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac09ac-159c-4294-a216-e4323acbfafc None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:08.825443 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac09ac-159c-4294-a216-e4323acbfafc None admin] PATH_INFO: `/v3/projects/cfff177a8165466dac56ca0ec7923f3e` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:08.827135 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdac09ac-159c-4294-a216-e4323acbfafc None admin] RBAC: Authorizing `identity:get_project(project_id=cfff177a8165466dac56ca0ec7923f3e)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:08.828730 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdac09ac-159c-4294-a216-e4323acbfafc None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:08.830568 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 533/1075] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:03:08 2022] GET /identity/v3/projects/cfff177a8165466dac56ca0ec7923f3e => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:08.831850 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdac09ac-159c-4294-a216-e4323acbfafc None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:08.867354 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040cd814-517f-4057-992e-3b67dfce052f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:08.899085 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:08.916741 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea1026-0e7e-4b90-b786-24b118b6730f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:08.917501 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea1026-0e7e-4b90-b786-24b118b6730f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:08.917857 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea1026-0e7e-4b90-b786-24b118b6730f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:08.918206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea1026-0e7e-4b90-b786-24b118b6730f None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:08.919073 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea1026-0e7e-4b90-b786-24b118b6730f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:03:08.919458 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea1026-0e7e-4b90-b786-24b118b6730f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:08.920932 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea1026-0e7e-4b90-b786-24b118b6730f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:08.933535 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 543/1076] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jul 13 23:03:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:08.934163 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea1026-0e7e-4b90-b786-24b118b6730f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:08.951684 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdac09ac-159c-4294-a216-e4323acbfafc None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:08.988440 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12905133-e9b8-490c-95a5-25e2f0bd86c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:08.989252 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12905133-e9b8-490c-95a5-25e2f0bd86c2 None admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:08.989647 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12905133-e9b8-490c-95a5-25e2f0bd86c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:08.990009 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12905133-e9b8-490c-95a5-25e2f0bd86c2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:08.990832 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12905133-e9b8-490c-95a5-25e2f0bd86c2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:08.992383 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12905133-e9b8-490c-95a5-25e2f0bd86c2 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:09.012611 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 534/1077] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Jul 13 23:03:08 2022] POST /identity/v3/endpoints => generated 327 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:03:09.016603 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12905133-e9b8-490c-95a5-25e2f0bd86c2 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:09.050949 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea1026-0e7e-4b90-b786-24b118b6730f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:09.078264 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:09.114499 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7d1e31-a111-4dad-92bb-2594a944f25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:09.115986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d1e31-a111-4dad-92bb-2594a944f25a None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:09.116460 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d1e31-a111-4dad-92bb-2594a944f25a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:09.116822 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7d1e31-a111-4dad-92bb-2594a944f25a None admin] PATH_INFO: `/v3/projects/cfff177a8165466dac56ca0ec7923f3e/users/fa93f188af7945a08840d5dda10d52a8/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:09.119153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d1e31-a111-4dad-92bb-2594a944f25a None admin] RBAC: Authorizing `identity:create_grant(project_id=cfff177a8165466dac56ca0ec7923f3e, user_id=fa93f188af7945a08840d5dda10d52a8, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:09.120860 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7d1e31-a111-4dad-92bb-2594a944f25a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:09.155772 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 544/1078] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jul 13 23:03:08 2022] PUT /identity/v3/projects/cfff177a8165466dac56ca0ec7923f3e/users/fa93f188af7945a08840d5dda10d52a8/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:03:09.166918 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12905133-e9b8-490c-95a5-25e2f0bd86c2 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:09.213912 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f90e29-6b89-4aa0-bf81-8d289e3f2a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:09.214910 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f90e29-6b89-4aa0-bf81-8d289e3f2a17 None admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:09.215234 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f90e29-6b89-4aa0-bf81-8d289e3f2a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:09.215639 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f90e29-6b89-4aa0-bf81-8d289e3f2a17 None admin] PATH_INFO: `/v3/projects/ab0a5581e4e444c293815e1304e20c05/users/4789384d158b4995bd30e1e37e30f5f1/roles/f7411bbcd41a4f959f76b130485bfb89` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:09.219389 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f90e29-6b89-4aa0-bf81-8d289e3f2a17 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab0a5581e4e444c293815e1304e20c05, user_id=4789384d158b4995bd30e1e37e30f5f1, role_id=f7411bbcd41a4f959f76b130485bfb89)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:09.221687 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f90e29-6b89-4aa0-bf81-8d289e3f2a17 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:09.241842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 535/1079] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jul 13 23:03:09 2022] PUT /identity/v3/projects/ab0a5581e4e444c293815e1304e20c05/users/4789384d158b4995bd30e1e37e30f5f1/roles/f7411bbcd41a4f959f76b130485bfb89 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:03:11.239516 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71553fce-cf9b-464a-a668-61f03e2dbfc0 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:11.240386 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71553fce-cf9b-464a-a668-61f03e2dbfc0 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:11.240386 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71553fce-cf9b-464a-a668-61f03e2dbfc0 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:11.241835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 545/1080] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:03:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:03:11.251224 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955870f-6eff-41e7-8a84-846fb808fc97 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:11.251813 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955870f-6eff-41e7-8a84-846fb808fc97 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:11.252224 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955870f-6eff-41e7-8a84-846fb808fc97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:11.276639 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-8955870f-6eff-41e7-8a84-846fb808fc97 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:03:11.277912 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-8955870f-6eff-41e7-8a84-846fb808fc97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:03:11.371591 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 536/1081] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:03:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:03:11.414510 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71553fce-cf9b-464a-a668-61f03e2dbfc0 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:11.479986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:11.600956 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71553fce-cf9b-464a-a668-61f03e2dbfc0 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:11.660881 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369894f3-4a70-406f-a67e-027a8d36167d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:11.662356 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369894f3-4a70-406f-a67e-027a8d36167d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:11.662870 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369894f3-4a70-406f-a67e-027a8d36167d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:11.663365 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369894f3-4a70-406f-a67e-027a8d36167d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:11.683588 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369894f3-4a70-406f-a67e-027a8d36167d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:03:11.698865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-369894f3-4a70-406f-a67e-027a8d36167d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:03:11.698865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:03:11.698865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:03:11.698865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:03:11.698865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:03:11.698865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:03:11.698865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:03:11.698865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:03:11.698865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:03:11.698865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:03:11.698865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:03:11.698865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:03:11.698865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:03:11.698865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:03:11.698865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:03:11.698865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:03:11.699889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:03:11.699889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:03:11.699889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:03:11.699889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:03:11.699889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:03:11.699889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:03:11.699889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:03:11.699889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:03:11.699889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:03:11.699889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:03:11.699889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:03:11.699889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:03:11.699889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:03:11.699889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:03:11.699889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:03:11.699889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:03:11.700738 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:03:11.700738 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:03:11.700738 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:03:11.700738 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:03:11.700738 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:03:11.700738 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:03:11.700738 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:03:11.700738 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:03:11.700738 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:03:11.700738 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:03:11.700738 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:03:11.700738 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:03:11.703880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 546/1082] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jul 13 23:03:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:03:11.711291 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955870f-6eff-41e7-8a84-846fb808fc97 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:11.747405 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:11.833378 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955870f-6eff-41e7-8a84-846fb808fc97 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:11.873346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75edc0d-ac5b-4315-a9d6-96e0ee172ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:11.873905 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75edc0d-ac5b-4315-a9d6-96e0ee172ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:11.874182 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75edc0d-ac5b-4315-a9d6-96e0ee172ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:11.874449 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75edc0d-ac5b-4315-a9d6-96e0ee172ca0 None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:11.875223 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75edc0d-ac5b-4315-a9d6-96e0ee172ca0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:03:11.875524 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75edc0d-ac5b-4315-a9d6-96e0ee172ca0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:11.876939 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75edc0d-ac5b-4315-a9d6-96e0ee172ca0 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:11.886778 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 537/1083] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:03:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:03:11.893081 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369894f3-4a70-406f-a67e-027a8d36167d None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:12.056171 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:12.068007 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369894f3-4a70-406f-a67e-027a8d36167d None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:12.123321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad9c94-3b0a-454a-b554-d0be648b2eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:12.125013 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad9c94-3b0a-454a-b554-d0be648b2eab None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:12.125518 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad9c94-3b0a-454a-b554-d0be648b2eab None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:12.126012 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad9c94-3b0a-454a-b554-d0be648b2eab None admin] PATH_INFO: `/v3/users/fa93f188af7945a08840d5dda10d52a8` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:12.128192 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aad9c94-3b0a-454a-b554-d0be648b2eab None admin] RBAC: Authorizing `identity:get_user(user_id=fa93f188af7945a08840d5dda10d52a8)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:12.131357 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aad9c94-3b0a-454a-b554-d0be648b2eab None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:12.139179 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 547/1084] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:03:11 2022] GET /identity/v3/users/fa93f188af7945a08840d5dda10d52a8 => generated 285 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:03:12.140581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75edc0d-ac5b-4315-a9d6-96e0ee172ca0 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:12.249261 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:12.259561 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75edc0d-ac5b-4315-a9d6-96e0ee172ca0 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:12.297625 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f4863-62ca-4259-9f51-198350463614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:12.298285 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f4863-62ca-4259-9f51-198350463614 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:12.298589 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f4863-62ca-4259-9f51-198350463614 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:12.298840 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f4863-62ca-4259-9f51-198350463614 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:12.386907 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20f4863-62ca-4259-9f51-198350463614 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:12.388337 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20f4863-62ca-4259-9f51-198350463614 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:12.413705 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0c697-6bf6-4b86-b522-6852e9f5414d None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:12.414450 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0c697-6bf6-4b86-b522-6852e9f5414d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:12.414450 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda0c697-6bf6-4b86-b522-6852e9f5414d None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:12.415968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 548/1085] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:03:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:03:12.425500 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb90584-0a33-40b1-bb3b-88118af99735 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:12.426909 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb90584-0a33-40b1-bb3b-88118af99735 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:12.427730 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb90584-0a33-40b1-bb3b-88118af99735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:12.455853 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-dbb90584-0a33-40b1-bb3b-88118af99735 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:03:12.458427 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-dbb90584-0a33-40b1-bb3b-88118af99735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:03:12.537140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:12.547355 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 538/1086] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:03:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 23:03:12.553983 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f4863-62ca-4259-9f51-198350463614 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:12.595566 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 549/1087] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:03:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:03:12.627510 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb90584-0a33-40b1-bb3b-88118af99735 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:12.684332 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f4863-62ca-4259-9f51-198350463614 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:12.733209 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea8a6b-f891-4c88-ba72-ad29edb8c6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:12.734075 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea8a6b-f891-4c88-ba72-ad29edb8c6d7 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:12.734358 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea8a6b-f891-4c88-ba72-ad29edb8c6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:12.734640 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea8a6b-f891-4c88-ba72-ad29edb8c6d7 None admin] PATH_INFO: `/v3/projects/cfff177a8165466dac56ca0ec7923f3e` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:12.735900 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea8a6b-f891-4c88-ba72-ad29edb8c6d7 None admin] RBAC: Authorizing `identity:get_project(project_id=cfff177a8165466dac56ca0ec7923f3e)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:12.737367 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea8a6b-f891-4c88-ba72-ad29edb8c6d7 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:12.739725 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 539/1088] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:03:12 2022] GET /identity/v3/projects/cfff177a8165466dac56ca0ec7923f3e => generated 311 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:12.741029 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3a9a8-8248-451b-9282-4f7a96aa9ee2 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:12.741318 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3a9a8-8248-451b-9282-4f7a96aa9ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:12.741597 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3a9a8-8248-451b-9282-4f7a96aa9ee2 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:12.743565 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 540/1089] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:03:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:03:12.746404 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3a9a8-8248-451b-9282-4f7a96aa9ee2 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:12.754119 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:12.765229 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb90584-0a33-40b1-bb3b-88118af99735 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:12.816543 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3426f-bf07-4b24-bff7-b5effd424c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:12.817559 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3426f-bf07-4b24-bff7-b5effd424c9a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:12.817989 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3426f-bf07-4b24-bff7-b5effd424c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:12.818395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3426f-bf07-4b24-bff7-b5effd424c9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:12.838407 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:12.891799 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3a9a8-8248-451b-9282-4f7a96aa9ee2 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:12.911368 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3426f-bf07-4b24-bff7-b5effd424c9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:12.913323 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3426f-bf07-4b24-bff7-b5effd424c9a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:12.937912 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09fc98d-9fdc-4257-98ac-31b8c4f556b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:12.938876 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09fc98d-9fdc-4257-98ac-31b8c4f556b4 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:12.939213 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09fc98d-9fdc-4257-98ac-31b8c4f556b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:12.939699 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09fc98d-9fdc-4257-98ac-31b8c4f556b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:12.941190 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09fc98d-9fdc-4257-98ac-31b8c4f556b4 None admin] RBAC: Adding query filter params (role.id=8cfd0b6e71e14e0983c3eb4d7478e027, scope.project.id=cfff177a8165466dac56ca0ec7923f3e, user.id=fa93f188af7945a08840d5dda10d52a8) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:03:12.941717 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09fc98d-9fdc-4257-98ac-31b8c4f556b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:12.944394 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09fc98d-9fdc-4257-98ac-31b8c4f556b4 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:12.954220 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 541/1090] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:03:12 2022] GET /identity/v3/role_assignments?user.id=fa93f188af7945a08840d5dda10d52a8&scope.project.id=cfff177a8165466dac56ca0ec7923f3e&role.id=8cfd0b6e71e14e0983c3eb4d7478e027 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:03:12.955704 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086abb96-5236-4f0f-87b9-f2e7fe5077a0 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:12.956031 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086abb96-5236-4f0f-87b9-f2e7fe5077a0 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:12.956411 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086abb96-5236-4f0f-87b9-f2e7fe5077a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:12.979699 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-086abb96-5236-4f0f-87b9-f2e7fe5077a0 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:03:12.981665 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-086abb96-5236-4f0f-87b9-f2e7fe5077a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:03:13.081611 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:13.104050 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 550/1091] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:03:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 13 23:03:13.104294 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 542/1092] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:03:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:03:13.112734 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3426f-bf07-4b24-bff7-b5effd424c9a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:13.136692 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086abb96-5236-4f0f-87b9-f2e7fe5077a0 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:13.235188 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:13.276116 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3426f-bf07-4b24-bff7-b5effd424c9a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:13.312989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086abb96-5236-4f0f-87b9-f2e7fe5077a0 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:13.333835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012f7c6-f20a-4318-9250-a3e2a8f61481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:13.335192 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012f7c6-f20a-4318-9250-a3e2a8f61481 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:13.335719 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012f7c6-f20a-4318-9250-a3e2a8f61481 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:13.336212 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012f7c6-f20a-4318-9250-a3e2a8f61481 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:13.357649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8012f7c6-f20a-4318-9250-a3e2a8f61481 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:13.359349 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8012f7c6-f20a-4318-9250-a3e2a8f61481 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:13.369110 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c49d7a-5ff2-4a2c-bedd-b80ff5616feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:13.370049 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c49d7a-5ff2-4a2c-bedd-b80ff5616feb None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:13.370322 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c49d7a-5ff2-4a2c-bedd-b80ff5616feb None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:13.370680 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c49d7a-5ff2-4a2c-bedd-b80ff5616feb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:13.384578 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c49d7a-5ff2-4a2c-bedd-b80ff5616feb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:03:13.393050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-b6c49d7a-5ff2-4a2c-bedd-b80ff5616feb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 23:03:13.393050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:03:13.393050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:03:13.393050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:03:13.393050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:03:13.393050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:03:13.393050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:03:13.393050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:03:13.393050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:03:13.393050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:03:13.393050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:03:13.393050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:03:13.393050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:03:13.393050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:03:13.393050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:03:13.393050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:03:13.395652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:03:13.395652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:03:13.395652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:03:13.395652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:03:13.395652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:03:13.395652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:03:13.395652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:03:13.395652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:03:13.395652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:03:13.395652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:03:13.395652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:03:13.395652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:03:13.395652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:03:13.395652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:03:13.395652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:03:13.395652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:03:13.397240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:03:13.397240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:03:13.397240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:03:13.397240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:03:13.397240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:03:13.397240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:03:13.397240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:03:13.397240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:03:13.397240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:03:13.397240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:03:13.397240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 23:03:13.397240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:03:13.397240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 543/1093] 10.222.0.60 () {56 vars in 1120 bytes} [Wed Jul 13 23:03:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:03:13.398608 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-8012f7c6-f20a-4318-9250-a3e2a8f61481 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:03:13.398608 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:03:13.398608 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:03:13.398608 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:03:13.398608 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:03:13.398608 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:03:13.398608 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:03:13.398608 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:03:13.398608 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:03:13.398608 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:03:13.398608 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:03:13.398608 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:03:13.398608 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:03:13.398608 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:03:13.398608 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:03:13.410868 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:03:13.410868 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:03:13.410868 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:03:13.410868 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:03:13.410868 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:03:13.410868 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:03:13.410868 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:03:13.410868 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:03:13.410868 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:03:13.410868 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:03:13.410868 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:03:13.410868 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:03:13.410868 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:03:13.410868 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:03:13.410868 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:03:13.410868 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:03:13.410868 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:03:13.413234 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:03:13.413234 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:03:13.413234 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:03:13.413234 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:03:13.413234 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 551/1094] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:03:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:03:13.415177 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c49d7a-5ff2-4a2c-bedd-b80ff5616feb None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:13.415431 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012f7c6-f20a-4318-9250-a3e2a8f61481 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:13.489830 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:13.553617 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c49d7a-5ff2-4a2c-bedd-b80ff5616feb None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:13.564905 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012f7c6-f20a-4318-9250-a3e2a8f61481 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:13.576790 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:13.602175 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f7069d-a465-47c6-b7b7-028a57cccc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:13.603223 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7069d-a465-47c6-b7b7-028a57cccc4f None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:13.603726 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7069d-a465-47c6-b7b7-028a57cccc4f None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:13.604295 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7069d-a465-47c6-b7b7-028a57cccc4f None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:13.605299 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f7069d-a465-47c6-b7b7-028a57cccc4f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:03:13.605880 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f7069d-a465-47c6-b7b7-028a57cccc4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:13.608084 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f7069d-a465-47c6-b7b7-028a57cccc4f None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:13.615269 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67fd48-6592-4a6f-970c-8ca6d94d1b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:13.616825 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67fd48-6592-4a6f-970c-8ca6d94d1b60 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:13.617276 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67fd48-6592-4a6f-970c-8ca6d94d1b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:13.617557 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67fd48-6592-4a6f-970c-8ca6d94d1b60 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:13.618306 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67fd48-6592-4a6f-970c-8ca6d94d1b60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:03:13.618616 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67fd48-6592-4a6f-970c-8ca6d94d1b60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:13.620148 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67fd48-6592-4a6f-970c-8ca6d94d1b60 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:13.620672 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 544/1095] 10.222.0.60 () {56 vars in 1115 bytes} [Wed Jul 13 23:03:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:13.626623 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f7069d-a465-47c6-b7b7-028a57cccc4f None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:13.634160 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 552/1096] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:03:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:03:13.764901 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f7069d-a465-47c6-b7b7-028a57cccc4f None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:13.808416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd25cc9-000f-41ea-8d8d-0e82299cd3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:13.809696 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd25cc9-000f-41ea-8d8d-0e82299cd3a5 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:13.810063 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd25cc9-000f-41ea-8d8d-0e82299cd3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:13.810484 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd25cc9-000f-41ea-8d8d-0e82299cd3a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:13.829207 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd25cc9-000f-41ea-8d8d-0e82299cd3a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:13.831789 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd25cc9-000f-41ea-8d8d-0e82299cd3a5 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:13.860036 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-7bd25cc9-000f-41ea-8d8d-0e82299cd3a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:03:13.860036 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:03:13.860036 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:03:13.860036 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:03:13.860036 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:03:13.860036 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:03:13.860036 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:03:13.860036 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:03:13.860036 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:03:13.860036 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:03:13.860036 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:03:13.860036 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:03:13.860036 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:03:13.860036 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:03:13.860036 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:03:13.860989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:03:13.860989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:03:13.860989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:03:13.860989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:03:13.860989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:03:13.860989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:03:13.860989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:03:13.860989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:03:13.860989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:03:13.860989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:03:13.860989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:03:13.860989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:03:13.860989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:03:13.860989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:03:13.860989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:03:13.860989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:03:13.860989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:03:13.861912 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:03:13.861912 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:03:13.861912 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:03:13.861912 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:03:13.862720 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 545/1097] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:03:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:03:13.869248 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67fd48-6592-4a6f-970c-8ca6d94d1b60 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:14.006939 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67fd48-6592-4a6f-970c-8ca6d94d1b60 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:14.014606 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:14.043975 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1812c75b-d0e8-4e4d-a6f9-620f7d116c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:14.044628 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812c75b-d0e8-4e4d-a6f9-620f7d116c01 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:14.044916 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812c75b-d0e8-4e4d-a6f9-620f7d116c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:14.045329 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812c75b-d0e8-4e4d-a6f9-620f7d116c01 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:14.046404 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1812c75b-d0e8-4e4d-a6f9-620f7d116c01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:03:14.046714 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1812c75b-d0e8-4e4d-a6f9-620f7d116c01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:14.048153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1812c75b-d0e8-4e4d-a6f9-620f7d116c01 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:14.062938 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 553/1098] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:03:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:03:14.069704 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd25cc9-000f-41ea-8d8d-0e82299cd3a5 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:14.102290 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:14.178357 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd25cc9-000f-41ea-8d8d-0e82299cd3a5 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:14.216128 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e3444-5d23-4e83-943f-35b1db8bf334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:14.216927 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e3444-5d23-4e83-943f-35b1db8bf334 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:14.217211 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e3444-5d23-4e83-943f-35b1db8bf334 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:14.217484 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e3444-5d23-4e83-943f-35b1db8bf334 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:14.226536 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9e3444-5d23-4e83-943f-35b1db8bf334 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:14.228067 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9e3444-5d23-4e83-943f-35b1db8bf334 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:14.242803 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-ed9e3444-5d23-4e83-943f-35b1db8bf334 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 23:03:14.242803 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:03:14.242803 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:03:14.242803 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:03:14.242803 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:03:14.242803 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:03:14.242803 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:03:14.242803 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:03:14.242803 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:03:14.242803 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:03:14.242803 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:03:14.242803 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:03:14.242803 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 23:03:14.242803 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 23:03:14.242803 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 23:03:14.242803 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 23:03:14.244686 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:03:14.244686 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:03:14.244686 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 23:03:14.244686 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:03:14.244686 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 23:03:14.244686 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:03:14.244686 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:03:14.244686 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:03:14.244686 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:03:14.244686 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:03:14.244686 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:03:14.244686 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:03:14.244686 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:03:14.244686 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:03:14.244686 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:03:14.244686 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:03:14.244686 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:03:14.244686 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:03:14.247078 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:03:14.247078 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:03:14.247078 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 23:03:14.247078 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 23:03:14.247078 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 23:03:14.247078 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 23:03:14.247078 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 23:03:14.247078 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 23:03:14.247078 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 23:03:14.247078 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:03:14.247078 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 546/1099] 10.222.0.60 () {56 vars in 1093 bytes} [Wed Jul 13 23:03:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:03:14.251849 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1812c75b-d0e8-4e4d-a6f9-620f7d116c01 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:14.358628 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1812c75b-d0e8-4e4d-a6f9-620f7d116c01 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:14.400040 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75892b4e-806d-41c7-8d42-d1acae09023c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:14.400714 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75892b4e-806d-41c7-8d42-d1acae09023c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:14.400997 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75892b4e-806d-41c7-8d42-d1acae09023c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:14.401274 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75892b4e-806d-41c7-8d42-d1acae09023c None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:14.402128 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75892b4e-806d-41c7-8d42-d1acae09023c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:03:14.402441 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75892b4e-806d-41c7-8d42-d1acae09023c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:14.403932 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75892b4e-806d-41c7-8d42-d1acae09023c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:14.422871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 554/1100] 10.222.0.60 () {56 vars in 1133 bytes} [Wed Jul 13 23:03:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:03:14.429272 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e3444-5d23-4e83-943f-35b1db8bf334 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:14.526354 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:14.536528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e3444-5d23-4e83-943f-35b1db8bf334 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:14.572896 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e0563-5cc9-45ad-bedb-12c719d52274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:14.573567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e0563-5cc9-45ad-bedb-12c719d52274 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:14.573854 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e0563-5cc9-45ad-bedb-12c719d52274 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:14.574135 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e0563-5cc9-45ad-bedb-12c719d52274 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:14.648459 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22e0563-5cc9-45ad-bedb-12c719d52274 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:14.649941 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22e0563-5cc9-45ad-bedb-12c719d52274 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:14.748856 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:14.760479 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 547/1101] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:03:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 23:03:14.766671 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75892b4e-806d-41c7-8d42-d1acae09023c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:14.793934 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:14.881664 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75892b4e-806d-41c7-8d42-d1acae09023c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:14.917513 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b7bd0-4570-448f-8db8-880deede6945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:14.918319 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b7bd0-4570-448f-8db8-880deede6945 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:14.918605 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b7bd0-4570-448f-8db8-880deede6945 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:14.918882 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b7bd0-4570-448f-8db8-880deede6945 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:14.930731 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b7bd0-4570-448f-8db8-880deede6945 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:14.932243 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b7bd0-4570-448f-8db8-880deede6945 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:14.952245 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-337b7bd0-4570-448f-8db8-880deede6945 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:03:14.952245 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:03:14.952245 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:03:14.952245 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:03:14.952245 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:03:14.952245 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:03:14.952245 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:03:14.952245 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:03:14.952245 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:03:14.952245 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:03:14.952245 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:03:14.952245 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:03:14.952245 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:03:14.952245 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:03:14.952245 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:03:14.953957 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:03:14.953957 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:03:14.953957 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:03:14.953957 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:03:14.953957 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:03:14.953957 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:03:14.953957 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:03:14.953957 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:03:14.953957 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:03:14.953957 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:03:14.953957 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:03:14.953957 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:03:14.953957 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:03:14.953957 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:03:14.953957 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:03:14.953957 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:03:14.953957 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:03:14.954937 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:03:14.954937 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:03:14.954937 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:03:14.954937 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:03:14.954937 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 555/1102] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:03:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:03:14.959437 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e0563-5cc9-45ad-bedb-12c719d52274 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:15.066266 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e0563-5cc9-45ad-bedb-12c719d52274 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:15.105317 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a31e87-51e0-4975-a314-26a07590194f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:15.106015 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a31e87-51e0-4975-a314-26a07590194f None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:15.106302 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a31e87-51e0-4975-a314-26a07590194f None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:15.106580 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a31e87-51e0-4975-a314-26a07590194f None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:15.107344 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a31e87-51e0-4975-a314-26a07590194f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:03:15.107695 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a31e87-51e0-4975-a314-26a07590194f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:15.109156 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a31e87-51e0-4975-a314-26a07590194f None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:15.123293 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 548/1103] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:03:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:15.129159 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b7bd0-4570-448f-8db8-880deede6945 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:15.224483 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:15.254915 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b7bd0-4570-448f-8db8-880deede6945 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:15.294413 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8325dd4-3dad-4487-8504-6f61fcb70d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:15.295223 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8325dd4-3dad-4487-8504-6f61fcb70d00 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:15.295503 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8325dd4-3dad-4487-8504-6f61fcb70d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:15.295884 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8325dd4-3dad-4487-8504-6f61fcb70d00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:15.304707 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8325dd4-3dad-4487-8504-6f61fcb70d00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:15.306241 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8325dd4-3dad-4487-8504-6f61fcb70d00 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:15.322435 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-f8325dd4-3dad-4487-8504-6f61fcb70d00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:03:15.322435 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:03:15.322435 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:03:15.322435 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:03:15.322435 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:03:15.322435 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:03:15.322435 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:03:15.322435 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:03:15.322435 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:03:15.322435 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:03:15.322435 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:03:15.322435 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:03:15.322435 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 23:03:15.322435 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 23:03:15.322435 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 23:03:15.324462 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 23:03:15.324462 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:03:15.324462 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:03:15.324462 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:03:15.324462 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:03:15.324462 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:03:15.324462 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:03:15.324462 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:03:15.324462 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:03:15.324462 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:03:15.324462 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:03:15.324462 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:03:15.324462 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:03:15.324462 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:03:15.324462 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:03:15.324462 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:03:15.324462 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:03:15.326421 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 23:03:15.326421 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 23:03:15.326421 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 23:03:15.326421 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 23:03:15.326421 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 23:03:15.326421 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 23:03:15.326421 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:03:15.326421 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:03:15.327826 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 556/1104] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:03:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:03:15.330401 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a31e87-51e0-4975-a314-26a07590194f None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:15.406490 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:15.470553 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a31e87-51e0-4975-a314-26a07590194f None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:15.512304 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02689ed-24d4-44f0-8723-826a1baaa23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:15.512879 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02689ed-24d4-44f0-8723-826a1baaa23d None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:15.513158 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02689ed-24d4-44f0-8723-826a1baaa23d None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:15.513428 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02689ed-24d4-44f0-8723-826a1baaa23d None admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:15.514249 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02689ed-24d4-44f0-8723-826a1baaa23d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:03:15.514555 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02689ed-24d4-44f0-8723-826a1baaa23d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:15.516101 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02689ed-24d4-44f0-8723-826a1baaa23d None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:15.531187 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 549/1105] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jul 13 23:03:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:03:15.538094 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8325dd4-3dad-4487-8504-6f61fcb70d00 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:15.654718 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8325dd4-3dad-4487-8504-6f61fcb70d00 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:15.693752 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013308d1-e55c-474d-a871-b1e4741d0933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:15.694671 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013308d1-e55c-474d-a871-b1e4741d0933 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:15.694953 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013308d1-e55c-474d-a871-b1e4741d0933 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:15.695227 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013308d1-e55c-474d-a871-b1e4741d0933 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:15.696111 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013308d1-e55c-474d-a871-b1e4741d0933 None admin] RBAC: Adding query filter params (role.id=f7411bbcd41a4f959f76b130485bfb89, scope.project.id=ab0a5581e4e444c293815e1304e20c05, user.id=4789384d158b4995bd30e1e37e30f5f1) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:03:15.696423 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013308d1-e55c-474d-a871-b1e4741d0933 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:15.697892 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013308d1-e55c-474d-a871-b1e4741d0933 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:15.701458 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:15.706214 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 557/1106] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:03:15 2022] GET /identity/v3/role_assignments?user.id=4789384d158b4995bd30e1e37e30f5f1&scope.project.id=ab0a5581e4e444c293815e1304e20c05&role.id=f7411bbcd41a4f959f76b130485bfb89 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:15.967624 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310758a-68c0-4e70-b665-9293e290c972 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:15.968024 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310758a-68c0-4e70-b665-9293e290c972 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:15.968377 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310758a-68c0-4e70-b665-9293e290c972 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:15.969908 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 550/1107] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:03:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:03:15.977181 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342fd190-8e9b-4b69-abb3-56ec80d4a38b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:15.977666 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342fd190-8e9b-4b69-abb3-56ec80d4a38b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:15.978094 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342fd190-8e9b-4b69-abb3-56ec80d4a38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:16.000599 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-342fd190-8e9b-4b69-abb3-56ec80d4a38b None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:03:16.002424 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-342fd190-8e9b-4b69-abb3-56ec80d4a38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:03:16.101029 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 558/1108] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:03:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:03:16.134353 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0310758a-68c0-4e70-b665-9293e290c972 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:16.277737 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0310758a-68c0-4e70-b665-9293e290c972 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:16.290969 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:16.318328 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf8fb8e-5e90-4888-8722-17fac379f163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:16.319389 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf8fb8e-5e90-4888-8722-17fac379f163 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:16.319706 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf8fb8e-5e90-4888-8722-17fac379f163 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:16.319988 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf8fb8e-5e90-4888-8722-17fac379f163 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:16.321206 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf8fb8e-5e90-4888-8722-17fac379f163 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:16.322674 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf8fb8e-5e90-4888-8722-17fac379f163 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:16.324438 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 551/1109] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:03:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:16.331119 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342fd190-8e9b-4b69-abb3-56ec80d4a38b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:16.459792 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342fd190-8e9b-4b69-abb3-56ec80d4a38b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:16.502418 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:16.512314 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28520a9f-535c-4036-9dbe-dd282f65ebd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:16.513420 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28520a9f-535c-4036-9dbe-dd282f65ebd5 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:16.513904 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28520a9f-535c-4036-9dbe-dd282f65ebd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:16.514371 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28520a9f-535c-4036-9dbe-dd282f65ebd5 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:16.515207 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28520a9f-535c-4036-9dbe-dd282f65ebd5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:16.517724 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28520a9f-535c-4036-9dbe-dd282f65ebd5 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:16.548719 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 559/1110] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Jul 13 23:03:16 2022] POST /identity/v3/users => generated 286 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:03:18.716987 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c551dee-0cd9-4f3f-8ca3-97843055b736 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:18.717664 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c551dee-0cd9-4f3f-8ca3-97843055b736 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:18.717664 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c551dee-0cd9-4f3f-8ca3-97843055b736 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:18.718916 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 552/1111] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:03:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:03:18.725245 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165bd040-777b-41cb-a09b-667194a538e5 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:18.725800 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165bd040-777b-41cb-a09b-667194a538e5 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:18.725800 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165bd040-777b-41cb-a09b-667194a538e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:18.743201 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-165bd040-777b-41cb-a09b-667194a538e5 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:03:18.744436 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-165bd040-777b-41cb-a09b-667194a538e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:03:18.813682 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 560/1112] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:03:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 23:03:18.838289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c551dee-0cd9-4f3f-8ca3-97843055b736 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:18.954732 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c551dee-0cd9-4f3f-8ca3-97843055b736 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:18.997309 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de4ac03-3523-4629-a974-8e71d4acc938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:18.998111 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4ac03-3523-4629-a974-8e71d4acc938 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:18.998396 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4ac03-3523-4629-a974-8e71d4acc938 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:18.998673 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4ac03-3523-4629-a974-8e71d4acc938 None admin] PATH_INFO: `/v3/roles/5f0ca8371a394d339621748b570e78ee` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:19.000335 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de4ac03-3523-4629-a974-8e71d4acc938 None admin] RBAC: Authorizing `identity:get_role(role_id=5f0ca8371a394d339621748b570e78ee)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:19.001761 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de4ac03-3523-4629-a974-8e71d4acc938 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:19.003236 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 553/1113] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:03:18 2022] GET /identity/v3/roles/5f0ca8371a394d339621748b570e78ee => generated 223 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:03:19.010070 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165bd040-777b-41cb-a09b-667194a538e5 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:19.183901 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165bd040-777b-41cb-a09b-667194a538e5 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:19.208211 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:19.229361 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e02b6-e251-4258-b521-9e5079b5b5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:19.230318 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e02b6-e251-4258-b521-9e5079b5b5fa None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:19.230619 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e02b6-e251-4258-b521-9e5079b5b5fa None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:19.230890 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e02b6-e251-4258-b521-9e5079b5b5fa None admin] PATH_INFO: `/v3/users/1330eb33e8be4910810be4a98101a6b1` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:19.246329 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8e02b6-e251-4258-b521-9e5079b5b5fa None admin] RBAC: Authorizing `identity:get_user(user_id=1330eb33e8be4910810be4a98101a6b1)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:19.247882 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8e02b6-e251-4258-b521-9e5079b5b5fa None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:19.249642 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 561/1114] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:03:19 2022] GET /identity/v3/users/1330eb33e8be4910810be4a98101a6b1 => generated 286 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:19.255799 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de4ac03-3523-4629-a974-8e71d4acc938 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:19.308610 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:19.368111 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de4ac03-3523-4629-a974-8e71d4acc938 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:19.406581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb33322-4ba9-4ff2-b32b-77e82061f76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:19.407249 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb33322-4ba9-4ff2-b32b-77e82061f76d None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:19.407566 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb33322-4ba9-4ff2-b32b-77e82061f76d None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:19.407843 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb33322-4ba9-4ff2-b32b-77e82061f76d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:19.485252 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb33322-4ba9-4ff2-b32b-77e82061f76d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:19.486736 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb33322-4ba9-4ff2-b32b-77e82061f76d None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:19.536566 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:19.594051 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 554/1115] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:03:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 23:03:19.600356 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e02b6-e251-4258-b521-9e5079b5b5fa None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:19.708503 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e02b6-e251-4258-b521-9e5079b5b5fa None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:19.745072 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064f645c-028d-45b3-ba79-2b9e6e58c1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:19.745963 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f645c-028d-45b3-ba79-2b9e6e58c1fb None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:19.746240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f645c-028d-45b3-ba79-2b9e6e58c1fb None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:19.746512 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f645c-028d-45b3-ba79-2b9e6e58c1fb None admin] PATH_INFO: `/v3/projects/cfff177a8165466dac56ca0ec7923f3e` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:19.747938 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064f645c-028d-45b3-ba79-2b9e6e58c1fb None admin] RBAC: Authorizing `identity:get_project(project_id=cfff177a8165466dac56ca0ec7923f3e)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:19.749432 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064f645c-028d-45b3-ba79-2b9e6e58c1fb None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:19.751265 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 562/1116] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:03:19 2022] GET /identity/v3/projects/cfff177a8165466dac56ca0ec7923f3e => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:03:19.758274 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb33322-4ba9-4ff2-b32b-77e82061f76d None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:19.865358 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb33322-4ba9-4ff2-b32b-77e82061f76d None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:19.903206 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17b19e8-e094-432c-8e32-3c9dc3af6ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:19.903897 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17b19e8-e094-432c-8e32-3c9dc3af6ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:19.904174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17b19e8-e094-432c-8e32-3c9dc3af6ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:19.904441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17b19e8-e094-432c-8e32-3c9dc3af6ea7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:19.905275 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17b19e8-e094-432c-8e32-3c9dc3af6ea7 None admin] RBAC: Adding query filter params (role.id=5f0ca8371a394d339621748b570e78ee, scope.project.id=cfff177a8165466dac56ca0ec7923f3e, user.id=1330eb33e8be4910810be4a98101a6b1) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:03:19.905582 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17b19e8-e094-432c-8e32-3c9dc3af6ea7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:19.907041 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17b19e8-e094-432c-8e32-3c9dc3af6ea7 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:19.913777 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 555/1117] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:03:19 2022] GET /identity/v3/role_assignments?user.id=1330eb33e8be4910810be4a98101a6b1&scope.project.id=cfff177a8165466dac56ca0ec7923f3e&role.id=5f0ca8371a394d339621748b570e78ee => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:03:22.043094 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97495736-a45e-4781-82b5-33ad89c97051 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:22.043744 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97495736-a45e-4781-82b5-33ad89c97051 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:22.043744 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97495736-a45e-4781-82b5-33ad89c97051 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:22.045071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 563/1118] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:03:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:03:22.051145 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f7ffc-019f-4f28-b5e1-5bbc5b9e9c1c None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:22.051738 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f7ffc-019f-4f28-b5e1-5bbc5b9e9c1c None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:22.051738 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f7ffc-019f-4f28-b5e1-5bbc5b9e9c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:22.069574 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-203f7ffc-019f-4f28-b5e1-5bbc5b9e9c1c None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:03:22.070776 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-203f7ffc-019f-4f28-b5e1-5bbc5b9e9c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:03:22.074078 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:22.144156 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 556/1119] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:03:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 23:03:22.166783 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97495736-a45e-4781-82b5-33ad89c97051 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:22.217060 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:22.280331 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97495736-a45e-4781-82b5-33ad89c97051 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:22.318233 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a09e237-998f-49c1-99c7-662334e3e43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:22.319366 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09e237-998f-49c1-99c7-662334e3e43b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:22.319676 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09e237-998f-49c1-99c7-662334e3e43b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:22.319951 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09e237-998f-49c1-99c7-662334e3e43b None admin] PATH_INFO: `/v3/roles/5f0ca8371a394d339621748b570e78ee` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:22.321434 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a09e237-998f-49c1-99c7-662334e3e43b None admin] RBAC: Authorizing `identity:get_role(role_id=5f0ca8371a394d339621748b570e78ee)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:22.322813 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a09e237-998f-49c1-99c7-662334e3e43b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:22.324248 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 564/1120] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:03:22 2022] GET /identity/v3/roles/5f0ca8371a394d339621748b570e78ee => generated 223 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:03:22.330445 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f7ffc-019f-4f28-b5e1-5bbc5b9e9c1c None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:22.438146 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f7ffc-019f-4f28-b5e1-5bbc5b9e9c1c None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:22.474747 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eadb58-9ab5-499e-a6e6-9e4f09275c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:22.475778 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eadb58-9ab5-499e-a6e6-9e4f09275c8c None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:22.476059 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eadb58-9ab5-499e-a6e6-9e4f09275c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:22.476330 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eadb58-9ab5-499e-a6e6-9e4f09275c8c None admin] PATH_INFO: `/v3/users/1330eb33e8be4910810be4a98101a6b1` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:22.477558 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eadb58-9ab5-499e-a6e6-9e4f09275c8c None admin] RBAC: Authorizing `identity:get_user(user_id=1330eb33e8be4910810be4a98101a6b1)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:22.479015 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eadb58-9ab5-499e-a6e6-9e4f09275c8c None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:22.480738 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 557/1121] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:03:22 2022] GET /identity/v3/users/1330eb33e8be4910810be4a98101a6b1 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:03:22.487473 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a09e237-998f-49c1-99c7-662334e3e43b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:22.662218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a09e237-998f-49c1-99c7-662334e3e43b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:22.682568 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:22.722901 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d39a2b-6da9-48e0-a804-26e0fc2e7ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:22.724573 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d39a2b-6da9-48e0-a804-26e0fc2e7ac5 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:22.725076 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d39a2b-6da9-48e0-a804-26e0fc2e7ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:22.725571 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d39a2b-6da9-48e0-a804-26e0fc2e7ac5 None admin] PATH_INFO: `/v3/projects/cfff177a8165466dac56ca0ec7923f3e` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:22.727788 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d39a2b-6da9-48e0-a804-26e0fc2e7ac5 None admin] RBAC: Authorizing `identity:get_project(project_id=cfff177a8165466dac56ca0ec7923f3e)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:22.730551 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d39a2b-6da9-48e0-a804-26e0fc2e7ac5 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:22.733909 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 565/1122] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:03:22 2022] GET /identity/v3/projects/cfff177a8165466dac56ca0ec7923f3e => generated 311 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:22.740572 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eadb58-9ab5-499e-a6e6-9e4f09275c8c None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:22.751886 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:22.857120 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eadb58-9ab5-499e-a6e6-9e4f09275c8c None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:22.892575 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe5806-15a1-4004-8009-d74edec25cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:22.893470 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe5806-15a1-4004-8009-d74edec25cc6 None admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:22.893748 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe5806-15a1-4004-8009-d74edec25cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:22.894518 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe5806-15a1-4004-8009-d74edec25cc6 None admin] PATH_INFO: `/v3/projects/cfff177a8165466dac56ca0ec7923f3e/users/1330eb33e8be4910810be4a98101a6b1/roles/5f0ca8371a394d339621748b570e78ee` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:22.896988 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbe5806-15a1-4004-8009-d74edec25cc6 None admin] RBAC: Authorizing `identity:create_grant(project_id=cfff177a8165466dac56ca0ec7923f3e, user_id=1330eb33e8be4910810be4a98101a6b1, role_id=5f0ca8371a394d339621748b570e78ee)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:22.898528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbe5806-15a1-4004-8009-d74edec25cc6 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:22.913060 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 558/1123] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jul 13 23:03:22 2022] PUT /identity/v3/projects/cfff177a8165466dac56ca0ec7923f3e/users/1330eb33e8be4910810be4a98101a6b1/roles/5f0ca8371a394d339621748b570e78ee => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:03:24.993310 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82079af-6f19-4ae7-a7e0-30d18ab3bfe4 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:24.993868 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82079af-6f19-4ae7-a7e0-30d18ab3bfe4 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:24.993961 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82079af-6f19-4ae7-a7e0-30d18ab3bfe4 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:24.995473 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 566/1124] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:03:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:03:25.001459 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc18948-a059-4489-9cda-293fae1e8d3e None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:25.002021 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc18948-a059-4489-9cda-293fae1e8d3e None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:25.002021 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc18948-a059-4489-9cda-293fae1e8d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:25.021751 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-dfc18948-a059-4489-9cda-293fae1e8d3e None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:03:25.023021 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-dfc18948-a059-4489-9cda-293fae1e8d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:03:25.092921 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:25.099984 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 559/1125] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:03:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:03:25.141602 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82079af-6f19-4ae7-a7e0-30d18ab3bfe4 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:25.263396 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82079af-6f19-4ae7-a7e0-30d18ab3bfe4 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:25.306070 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7311775e-050f-4466-89c2-8a6ff0895c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:25.306936 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311775e-050f-4466-89c2-8a6ff0895c88 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:25.307211 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311775e-050f-4466-89c2-8a6ff0895c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:25.307480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311775e-050f-4466-89c2-8a6ff0895c88 None admin] PATH_INFO: `/v3/roles/5f0ca8371a394d339621748b570e78ee` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:25.309092 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7311775e-050f-4466-89c2-8a6ff0895c88 None admin] RBAC: Authorizing `identity:get_role(role_id=5f0ca8371a394d339621748b570e78ee)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:25.310468 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7311775e-050f-4466-89c2-8a6ff0895c88 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:25.311944 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 567/1126] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:03:25 2022] GET /identity/v3/roles/5f0ca8371a394d339621748b570e78ee => generated 223 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:25.318470 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc18948-a059-4489-9cda-293fae1e8d3e None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:25.423080 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc18948-a059-4489-9cda-293fae1e8d3e None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:25.457702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13ce10d-354f-4fa9-8c8c-b54b9f521efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:25.458581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13ce10d-354f-4fa9-8c8c-b54b9f521efc None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:25.458867 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13ce10d-354f-4fa9-8c8c-b54b9f521efc None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:25.459144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13ce10d-354f-4fa9-8c8c-b54b9f521efc None admin] PATH_INFO: `/v3/users/1330eb33e8be4910810be4a98101a6b1` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:25.460400 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13ce10d-354f-4fa9-8c8c-b54b9f521efc None admin] RBAC: Authorizing `identity:get_user(user_id=1330eb33e8be4910810be4a98101a6b1)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:25.461873 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13ce10d-354f-4fa9-8c8c-b54b9f521efc None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:25.463588 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 560/1127] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:03:25 2022] GET /identity/v3/users/1330eb33e8be4910810be4a98101a6b1 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:25.469617 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7311775e-050f-4466-89c2-8a6ff0895c88 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:25.517795 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:25.573756 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7311775e-050f-4466-89c2-8a6ff0895c88 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:25.617170 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e10aa57-0041-4296-a2d0-59a2c8496a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:25.617824 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10aa57-0041-4296-a2d0-59a2c8496a9a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:25.618100 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10aa57-0041-4296-a2d0-59a2c8496a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:25.618376 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10aa57-0041-4296-a2d0-59a2c8496a9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:25.689123 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e10aa57-0041-4296-a2d0-59a2c8496a9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:25.690613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e10aa57-0041-4296-a2d0-59a2c8496a9a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:25.736574 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:25.791397 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 568/1128] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:03:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 23:03:25.797406 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13ce10d-354f-4fa9-8c8c-b54b9f521efc None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:25.866026 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:25.905366 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13ce10d-354f-4fa9-8c8c-b54b9f521efc None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:25.943234 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb08718-fe68-4860-92cf-61a6480bd83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:25.944379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb08718-fe68-4860-92cf-61a6480bd83b None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:25.944662 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb08718-fe68-4860-92cf-61a6480bd83b None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:25.944937 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb08718-fe68-4860-92cf-61a6480bd83b None admin] PATH_INFO: `/v3/projects/cfff177a8165466dac56ca0ec7923f3e` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:25.946132 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb08718-fe68-4860-92cf-61a6480bd83b None admin] RBAC: Authorizing `identity:get_project(project_id=cfff177a8165466dac56ca0ec7923f3e)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:25.947684 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb08718-fe68-4860-92cf-61a6480bd83b None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:25.949425 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 561/1129] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:03:25 2022] GET /identity/v3/projects/cfff177a8165466dac56ca0ec7923f3e => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:03:25.955612 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e10aa57-0041-4296-a2d0-59a2c8496a9a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:26.056098 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e10aa57-0041-4296-a2d0-59a2c8496a9a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:26.091955 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10cc9c-7c36-480a-aceb-b634e46ab59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:26.092609 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10cc9c-7c36-480a-aceb-b634e46ab59b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:26.092890 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10cc9c-7c36-480a-aceb-b634e46ab59b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:26.093166 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10cc9c-7c36-480a-aceb-b634e46ab59b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:26.093998 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e10cc9c-7c36-480a-aceb-b634e46ab59b None admin] RBAC: Adding query filter params (role.id=5f0ca8371a394d339621748b570e78ee, scope.project.id=cfff177a8165466dac56ca0ec7923f3e, user.id=1330eb33e8be4910810be4a98101a6b1) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:03:26.094309 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e10cc9c-7c36-480a-aceb-b634e46ab59b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:26.095774 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e10cc9c-7c36-480a-aceb-b634e46ab59b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:26.102337 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 569/1130] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:03:25 2022] GET /identity/v3/role_assignments?user.id=1330eb33e8be4910810be4a98101a6b1&scope.project.id=cfff177a8165466dac56ca0ec7923f3e&role.id=5f0ca8371a394d339621748b570e78ee => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:28.230501 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4db2fc-a82c-421f-8a86-6dff7b5e1bec None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:28.231073 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4db2fc-a82c-421f-8a86-6dff7b5e1bec None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:28.231073 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4db2fc-a82c-421f-8a86-6dff7b5e1bec None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:28.232471 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 562/1131] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:03:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:03:28.238094 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf29767-0c16-40e3-8dfd-9a9679db21ae None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:28.238559 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf29767-0c16-40e3-8dfd-9a9679db21ae None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:28.238723 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf29767-0c16-40e3-8dfd-9a9679db21ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:28.256171 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-baf29767-0c16-40e3-8dfd-9a9679db21ae None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:03:28.257389 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-baf29767-0c16-40e3-8dfd-9a9679db21ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:03:28.260593 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:28.328647 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 570/1132] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:03:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:03:28.353090 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4db2fc-a82c-421f-8a86-6dff7b5e1bec None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:28.521459 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4db2fc-a82c-421f-8a86-6dff7b5e1bec None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:28.577513 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b18642-2e49-45eb-a736-6e0cc19f0c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:28.578794 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b18642-2e49-45eb-a736-6e0cc19f0c57 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:28.579265 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b18642-2e49-45eb-a736-6e0cc19f0c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:28.579772 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b18642-2e49-45eb-a736-6e0cc19f0c57 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:28.582275 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b18642-2e49-45eb-a736-6e0cc19f0c57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:28.584839 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b18642-2e49-45eb-a736-6e0cc19f0c57 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:28.587858 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 563/1133] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:03:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:03:28.593855 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf29767-0c16-40e3-8dfd-9a9679db21ae None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:28.697598 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf29767-0c16-40e3-8dfd-9a9679db21ae None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:28.733672 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a186db55-2dd8-4de0-91a9-17f3eee4af08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:28.734328 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186db55-2dd8-4de0-91a9-17f3eee4af08 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:28.734610 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186db55-2dd8-4de0-91a9-17f3eee4af08 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:28.734890 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186db55-2dd8-4de0-91a9-17f3eee4af08 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:28.735622 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a186db55-2dd8-4de0-91a9-17f3eee4af08 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:28.737066 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a186db55-2dd8-4de0-91a9-17f3eee4af08 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:28.747378 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:28.765128 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 571/1134] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Jul 13 23:03:28 2022] POST /identity/v3/projects => generated 311 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:03:30.883629 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e428caf-0e42-4935-a6fc-afd61957c210 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:30.884420 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e428caf-0e42-4935-a6fc-afd61957c210 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:30.884420 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e428caf-0e42-4935-a6fc-afd61957c210 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:30.885539 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 564/1135] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:03:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:03:30.892478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b2a9b6-2be0-4295-9769-8c6f1b8e9b2a None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:30.893159 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b2a9b6-2be0-4295-9769-8c6f1b8e9b2a None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:30.894116 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b2a9b6-2be0-4295-9769-8c6f1b8e9b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:30.922431 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-42b2a9b6-2be0-4295-9769-8c6f1b8e9b2a None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:03:30.924595 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-42b2a9b6-2be0-4295-9769-8c6f1b8e9b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:03:31.052042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 572/1136] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:03:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:03:31.092985 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e428caf-0e42-4935-a6fc-afd61957c210 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:31.096098 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:31.236083 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e428caf-0e42-4935-a6fc-afd61957c210 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:31.274058 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21fc775-b81b-4d3b-a93f-e95d6e943510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:31.274910 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21fc775-b81b-4d3b-a93f-e95d6e943510 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:31.275244 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21fc775-b81b-4d3b-a93f-e95d6e943510 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:31.275653 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21fc775-b81b-4d3b-a93f-e95d6e943510 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:31.277020 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21fc775-b81b-4d3b-a93f-e95d6e943510 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:31.278710 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21fc775-b81b-4d3b-a93f-e95d6e943510 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:31.280721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 565/1137] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:03:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:03:31.287168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b2a9b6-2be0-4295-9769-8c6f1b8e9b2a None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:31.418602 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b2a9b6-2be0-4295-9769-8c6f1b8e9b2a None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:31.446500 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:31.461764 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6141c4-f9c3-4fc5-afb8-fbc0bf061f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:31.462508 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6141c4-f9c3-4fc5-afb8-fbc0bf061f80 None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:31.462835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6141c4-f9c3-4fc5-afb8-fbc0bf061f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:31.463152 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6141c4-f9c3-4fc5-afb8-fbc0bf061f80 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:31.464019 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6141c4-f9c3-4fc5-afb8-fbc0bf061f80 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:31.465606 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6141c4-f9c3-4fc5-afb8-fbc0bf061f80 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:31.490114 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 573/1138] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Jul 13 23:03:31 2022] POST /identity/v3/users => generated 286 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:03:33.669929 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722aa48b-e91d-4cdd-af7c-937febfaf095 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:33.670552 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722aa48b-e91d-4cdd-af7c-937febfaf095 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:33.670552 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722aa48b-e91d-4cdd-af7c-937febfaf095 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:33.672049 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 566/1139] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:03:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:03:33.678461 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da5923-d579-4804-a181-ffecc747de19 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:33.679006 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da5923-d579-4804-a181-ffecc747de19 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:33.679006 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da5923-d579-4804-a181-ffecc747de19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:33.698233 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-53da5923-d579-4804-a181-ffecc747de19 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:03:33.699505 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-53da5923-d579-4804-a181-ffecc747de19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:03:33.768972 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 574/1140] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:03:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 23:03:33.793892 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722aa48b-e91d-4cdd-af7c-937febfaf095 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:33.868203 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:33.902008 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722aa48b-e91d-4cdd-af7c-937febfaf095 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:33.938012 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98639445-3709-4776-9968-61ba9d004b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:33.938733 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98639445-3709-4776-9968-61ba9d004b3c None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:33.939020 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98639445-3709-4776-9968-61ba9d004b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:33.939296 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98639445-3709-4776-9968-61ba9d004b3c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:33.950277 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98639445-3709-4776-9968-61ba9d004b3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:03:33.958114 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-98639445-3709-4776-9968-61ba9d004b3c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:03:33.958114 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:03:33.958114 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:03:33.958114 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:03:33.958114 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:03:33.958114 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:03:33.958114 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:03:33.958114 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:03:33.958114 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:03:33.958114 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:03:33.958114 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:03:33.958114 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:03:33.958114 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:03:33.958114 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:03:33.958114 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:03:33.958114 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:03:33.960001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:03:33.960001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:03:33.960001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:03:33.960001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:03:33.960001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:03:33.960001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:03:33.960001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:03:33.960001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:03:33.960001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:03:33.960001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:03:33.960001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:03:33.960001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:03:33.960001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:03:33.960001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:03:33.960001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:03:33.960001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:03:33.961733 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:03:33.961733 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:03:33.961733 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:03:33.961733 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:03:33.961733 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:03:33.961733 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:03:33.961733 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:03:33.961733 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:03:33.961733 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:03:33.961733 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:03:33.961733 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:03:33.961733 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:03:33.961733 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 567/1141] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jul 13 23:03:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:03:33.965740 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53da5923-d579-4804-a181-ffecc747de19 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:34.116822 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53da5923-d579-4804-a181-ffecc747de19 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:34.127111 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:34.154711 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a26dd2-4c4c-406d-a4c0-31706c37a4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:34.155379 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a26dd2-4c4c-406d-a4c0-31706c37a4c7 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:34.155695 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a26dd2-4c4c-406d-a4c0-31706c37a4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:34.155965 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a26dd2-4c4c-406d-a4c0-31706c37a4c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:34.156716 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a26dd2-4c4c-406d-a4c0-31706c37a4c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:03:34.157017 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a26dd2-4c4c-406d-a4c0-31706c37a4c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:34.158384 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a26dd2-4c4c-406d-a4c0-31706c37a4c7 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:34.167786 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 575/1142] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:03:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:34.180470 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98639445-3709-4776-9968-61ba9d004b3c None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:34.294358 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98639445-3709-4776-9968-61ba9d004b3c None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:34.323616 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:34.330437 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e61a11e-2cf9-439e-bb23-fa5db216e46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:34.331980 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61a11e-2cf9-439e-bb23-fa5db216e46a None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:34.332382 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61a11e-2cf9-439e-bb23-fa5db216e46a None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:34.332749 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61a11e-2cf9-439e-bb23-fa5db216e46a None admin] PATH_INFO: `/v3/users/ae4c7fa8d4004382b2cd843c200af3a8` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:34.347719 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e61a11e-2cf9-439e-bb23-fa5db216e46a None admin] RBAC: Authorizing `identity:get_user(user_id=ae4c7fa8d4004382b2cd843c200af3a8)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:34.349203 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e61a11e-2cf9-439e-bb23-fa5db216e46a None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:34.350970 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 568/1143] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:03:34 2022] GET /identity/v3/users/ae4c7fa8d4004382b2cd843c200af3a8 => generated 286 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:34.357047 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a26dd2-4c4c-406d-a4c0-31706c37a4c7 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:34.461213 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a26dd2-4c4c-406d-a4c0-31706c37a4c7 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:34.504200 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac058a4a-66b2-4f52-8eac-2d0aada062bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:34.504849 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac058a4a-66b2-4f52-8eac-2d0aada062bb None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:34.505137 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac058a4a-66b2-4f52-8eac-2d0aada062bb None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:34.505415 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac058a4a-66b2-4f52-8eac-2d0aada062bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:34.546720 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:34.576925 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac058a4a-66b2-4f52-8eac-2d0aada062bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:34.578381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac058a4a-66b2-4f52-8eac-2d0aada062bb None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:34.679291 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 576/1144] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:03:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 23:03:34.685499 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e61a11e-2cf9-439e-bb23-fa5db216e46a None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:34.790859 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e61a11e-2cf9-439e-bb23-fa5db216e46a None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:34.825638 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd393501-f13d-4e86-8ac4-4014c22829f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:34.826411 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd393501-f13d-4e86-8ac4-4014c22829f0 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:34.826696 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd393501-f13d-4e86-8ac4-4014c22829f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:34.826967 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd393501-f13d-4e86-8ac4-4014c22829f0 None admin] PATH_INFO: `/v3/projects/7d41911da96e42ffa0e86c1f67b87121` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:34.828155 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd393501-f13d-4e86-8ac4-4014c22829f0 None admin] RBAC: Authorizing `identity:get_project(project_id=7d41911da96e42ffa0e86c1f67b87121)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:34.829587 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd393501-f13d-4e86-8ac4-4014c22829f0 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:34.831894 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 569/1145] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:03:34 2022] GET /identity/v3/projects/7d41911da96e42ffa0e86c1f67b87121 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:34.838543 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac058a4a-66b2-4f52-8eac-2d0aada062bb None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:34.920311 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:34.952459 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac058a4a-66b2-4f52-8eac-2d0aada062bb None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:34.986315 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24c0159-a740-49e8-a9ea-a50b91b28495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:34.987501 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c0159-a740-49e8-a9ea-a50b91b28495 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:34.987817 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c0159-a740-49e8-a9ea-a50b91b28495 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:34.988092 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c0159-a740-49e8-a9ea-a50b91b28495 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:34.988924 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24c0159-a740-49e8-a9ea-a50b91b28495 None admin] RBAC: Adding query filter params (role.id=8cfd0b6e71e14e0983c3eb4d7478e027, scope.project.id=7d41911da96e42ffa0e86c1f67b87121, user.id=ae4c7fa8d4004382b2cd843c200af3a8) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:03:34.989231 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24c0159-a740-49e8-a9ea-a50b91b28495 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:34.990689 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24c0159-a740-49e8-a9ea-a50b91b28495 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:34.996643 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 577/1146] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:03:34 2022] GET /identity/v3/role_assignments?user.id=ae4c7fa8d4004382b2cd843c200af3a8&scope.project.id=7d41911da96e42ffa0e86c1f67b87121&role.id=8cfd0b6e71e14e0983c3eb4d7478e027 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:37.032570 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a08e4-3516-4fc4-9878-9852f3fd2291 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:37.033182 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a08e4-3516-4fc4-9878-9852f3fd2291 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:37.033182 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a08e4-3516-4fc4-9878-9852f3fd2291 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:37.035291 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 570/1147] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:03:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:03:37.041074 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052cd4f-6bec-4505-8ece-6b3e0a6f5bd4 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:37.041607 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052cd4f-6bec-4505-8ece-6b3e0a6f5bd4 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:37.041607 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052cd4f-6bec-4505-8ece-6b3e0a6f5bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:37.059720 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-5052cd4f-6bec-4505-8ece-6b3e0a6f5bd4 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:03:37.060951 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5052cd4f-6bec-4505-8ece-6b3e0a6f5bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:03:37.133496 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 578/1148] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:03:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 23:03:37.156936 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6a08e4-3516-4fc4-9878-9852f3fd2291 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:37.220571 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:37.268703 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6a08e4-3516-4fc4-9878-9852f3fd2291 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:37.304062 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171f58a3-7617-436f-a6d2-79dfce85f937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:37.304768 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f58a3-7617-436f-a6d2-79dfce85f937 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:37.305049 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f58a3-7617-436f-a6d2-79dfce85f937 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:37.305321 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f58a3-7617-436f-a6d2-79dfce85f937 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:37.317765 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171f58a3-7617-436f-a6d2-79dfce85f937 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:03:37.325696 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-171f58a3-7617-436f-a6d2-79dfce85f937 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:03:37.325696 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:03:37.325696 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:03:37.325696 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:03:37.325696 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:03:37.325696 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:03:37.325696 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:03:37.325696 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:03:37.325696 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:03:37.325696 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:03:37.325696 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:03:37.325696 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:03:37.325696 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:03:37.325696 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:03:37.325696 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:03:37.325696 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:03:37.327418 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:03:37.327418 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:03:37.327418 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:03:37.327418 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:03:37.327418 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:03:37.327418 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:03:37.327418 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:03:37.327418 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:03:37.327418 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:03:37.327418 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:03:37.327418 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:03:37.327418 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:03:37.327418 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:03:37.327418 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:03:37.327418 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:03:37.327418 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:03:37.329117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:03:37.329117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:03:37.329117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:03:37.329117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:03:37.329117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:03:37.329117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:03:37.329117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:03:37.329117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:03:37.329117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:03:37.329117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:03:37.329117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:03:37.329117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:03:37.329117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 571/1149] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jul 13 23:03:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:03:37.335284 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5052cd4f-6bec-4505-8ece-6b3e0a6f5bd4 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:37.402578 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:37.443714 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5052cd4f-6bec-4505-8ece-6b3e0a6f5bd4 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:37.480742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da6ca10-2446-4fab-90e4-509c5d99de8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:37.481419 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6ca10-2446-4fab-90e4-509c5d99de8f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:37.481695 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6ca10-2446-4fab-90e4-509c5d99de8f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:37.481966 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6ca10-2446-4fab-90e4-509c5d99de8f None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:37.482732 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da6ca10-2446-4fab-90e4-509c5d99de8f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:03:37.483037 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da6ca10-2446-4fab-90e4-509c5d99de8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:37.484454 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da6ca10-2446-4fab-90e4-509c5d99de8f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:37.493676 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 579/1150] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:03:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:37.503903 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171f58a3-7617-436f-a6d2-79dfce85f937 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:37.674204 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171f58a3-7617-436f-a6d2-79dfce85f937 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:37.702981 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:37.733795 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b10125-3ba9-4b44-9e20-f3aa8252aefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:37.735332 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b10125-3ba9-4b44-9e20-f3aa8252aefd None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:37.735856 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b10125-3ba9-4b44-9e20-f3aa8252aefd None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:37.736339 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b10125-3ba9-4b44-9e20-f3aa8252aefd None admin] PATH_INFO: `/v3/users/ae4c7fa8d4004382b2cd843c200af3a8` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:37.738407 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b10125-3ba9-4b44-9e20-f3aa8252aefd None admin] RBAC: Authorizing `identity:get_user(user_id=ae4c7fa8d4004382b2cd843c200af3a8)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:37.741137 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b10125-3ba9-4b44-9e20-f3aa8252aefd None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:37.744374 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 572/1151] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:03:37 2022] GET /identity/v3/users/ae4c7fa8d4004382b2cd843c200af3a8 => generated 286 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:37.750415 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da6ca10-2446-4fab-90e4-509c5d99de8f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:37.856718 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da6ca10-2446-4fab-90e4-509c5d99de8f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:37.887115 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:37.897842 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d715260a-1c09-4363-af68-f38fc5d14e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:37.898713 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d715260a-1c09-4363-af68-f38fc5d14e56 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:37.898995 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d715260a-1c09-4363-af68-f38fc5d14e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:37.899267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d715260a-1c09-4363-af68-f38fc5d14e56 None admin] PATH_INFO: `/v3/projects/7d41911da96e42ffa0e86c1f67b87121` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:37.900543 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d715260a-1c09-4363-af68-f38fc5d14e56 None admin] RBAC: Authorizing `identity:get_project(project_id=7d41911da96e42ffa0e86c1f67b87121)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:37.901985 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d715260a-1c09-4363-af68-f38fc5d14e56 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:37.903814 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 580/1152] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:03:37 2022] GET /identity/v3/projects/7d41911da96e42ffa0e86c1f67b87121 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:03:37.910335 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b10125-3ba9-4b44-9e20-f3aa8252aefd None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:38.025252 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b10125-3ba9-4b44-9e20-f3aa8252aefd None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:38.061908 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab6300d-bf6c-4bba-8668-436b5f8d763b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:38.062981 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6300d-bf6c-4bba-8668-436b5f8d763b None admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:38.063151 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6300d-bf6c-4bba-8668-436b5f8d763b None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:38.063341 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab6300d-bf6c-4bba-8668-436b5f8d763b None admin] PATH_INFO: `/v3/projects/7d41911da96e42ffa0e86c1f67b87121/users/ae4c7fa8d4004382b2cd843c200af3a8/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:38.066097 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab6300d-bf6c-4bba-8668-436b5f8d763b None admin] RBAC: Authorizing `identity:create_grant(project_id=7d41911da96e42ffa0e86c1f67b87121, user_id=ae4c7fa8d4004382b2cd843c200af3a8, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:38.067582 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab6300d-bf6c-4bba-8668-436b5f8d763b None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:38.085000 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 573/1153] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jul 13 23:03:37 2022] PUT /identity/v3/projects/7d41911da96e42ffa0e86c1f67b87121/users/ae4c7fa8d4004382b2cd843c200af3a8/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:03:40.145890 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ffc728-0a8e-4b45-a237-f8f632280d24 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:40.146534 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ffc728-0a8e-4b45-a237-f8f632280d24 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:40.146534 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ffc728-0a8e-4b45-a237-f8f632280d24 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:40.148110 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 581/1154] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:03:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:03:40.154365 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c7b067-6221-4886-8332-3d05b3b75f57 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:40.154934 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c7b067-6221-4886-8332-3d05b3b75f57 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:40.155100 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c7b067-6221-4886-8332-3d05b3b75f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:40.173073 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-c6c7b067-6221-4886-8332-3d05b3b75f57 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:03:40.174372 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-c6c7b067-6221-4886-8332-3d05b3b75f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:03:40.218306 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:40.250265 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 574/1155] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:03:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:03:40.274914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ffc728-0a8e-4b45-a237-f8f632280d24 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:40.393662 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ffc728-0a8e-4b45-a237-f8f632280d24 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:40.427585 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0fa578-9115-4b3b-940d-5d8b951891d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:40.428277 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0fa578-9115-4b3b-940d-5d8b951891d4 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:40.428554 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0fa578-9115-4b3b-940d-5d8b951891d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:40.428825 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0fa578-9115-4b3b-940d-5d8b951891d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:40.439944 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0fa578-9115-4b3b-940d-5d8b951891d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:03:40.447776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-ba0fa578-9115-4b3b-940d-5d8b951891d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:03:40.447776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:03:40.447776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:03:40.447776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:03:40.447776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:03:40.447776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:03:40.447776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:03:40.447776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:03:40.447776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:03:40.447776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:03:40.447776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:03:40.447776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:03:40.447776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:03:40.447776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:03:40.447776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:03:40.447776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:03:40.449553 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:03:40.449553 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:03:40.449553 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:03:40.449553 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:03:40.449553 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:03:40.449553 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:03:40.449553 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:03:40.449553 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:03:40.449553 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:03:40.449553 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:03:40.449553 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:03:40.449553 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:03:40.449553 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:03:40.449553 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:03:40.449553 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:03:40.449553 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:03:40.451213 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:03:40.451213 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:03:40.451213 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:03:40.451213 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:03:40.451213 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:03:40.451213 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:03:40.451213 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:03:40.451213 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:03:40.451213 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:03:40.451213 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:03:40.451213 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:03:40.451213 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:03:40.451213 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 582/1156] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jul 13 23:03:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:03:40.458827 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c7b067-6221-4886-8332-3d05b3b75f57 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:40.562289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c7b067-6221-4886-8332-3d05b3b75f57 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:40.599259 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2150d5-c87a-479f-bfa7-8abdff5993b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:40.599972 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2150d5-c87a-479f-bfa7-8abdff5993b3 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:40.600258 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2150d5-c87a-479f-bfa7-8abdff5993b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:40.600537 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2150d5-c87a-479f-bfa7-8abdff5993b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:40.601301 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2150d5-c87a-479f-bfa7-8abdff5993b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:03:40.601618 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2150d5-c87a-479f-bfa7-8abdff5993b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:40.603028 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2150d5-c87a-479f-bfa7-8abdff5993b3 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:40.612390 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 575/1157] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:03:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:40.622012 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0fa578-9115-4b3b-940d-5d8b951891d4 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:40.720654 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:40.768631 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0fa578-9115-4b3b-940d-5d8b951891d4 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:40.804800 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58974eb3-ea4a-40e3-8774-6432cc4c8fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:40.805677 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58974eb3-ea4a-40e3-8774-6432cc4c8fad None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:40.805952 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58974eb3-ea4a-40e3-8774-6432cc4c8fad None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:40.806221 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58974eb3-ea4a-40e3-8774-6432cc4c8fad None admin] PATH_INFO: `/v3/users/ae4c7fa8d4004382b2cd843c200af3a8` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:40.807449 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58974eb3-ea4a-40e3-8774-6432cc4c8fad None admin] RBAC: Authorizing `identity:get_user(user_id=ae4c7fa8d4004382b2cd843c200af3a8)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:40.808958 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58974eb3-ea4a-40e3-8774-6432cc4c8fad None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:40.810698 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 583/1158] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:03:40 2022] GET /identity/v3/users/ae4c7fa8d4004382b2cd843c200af3a8 => generated 286 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:40.817240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2150d5-c87a-479f-bfa7-8abdff5993b3 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:40.854871 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:40.933556 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2150d5-c87a-479f-bfa7-8abdff5993b3 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:40.968468 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da6f9e-0cf3-49d0-8b3e-25891939a32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:40.969125 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da6f9e-0cf3-49d0-8b3e-25891939a32a None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:40.969410 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da6f9e-0cf3-49d0-8b3e-25891939a32a None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:40.969689 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da6f9e-0cf3-49d0-8b3e-25891939a32a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:41.044032 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67da6f9e-0cf3-49d0-8b3e-25891939a32a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:41.045441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67da6f9e-0cf3-49d0-8b3e-25891939a32a None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:41.083513 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:41.154556 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 576/1159] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:03:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 23:03:41.160994 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58974eb3-ea4a-40e3-8774-6432cc4c8fad None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:41.280043 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58974eb3-ea4a-40e3-8774-6432cc4c8fad None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:41.304155 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:41.316378 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240ad3ec-3499-44f8-b440-309fd0567e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:41.317271 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240ad3ec-3499-44f8-b440-309fd0567e1f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:41.317556 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240ad3ec-3499-44f8-b440-309fd0567e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:41.317833 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240ad3ec-3499-44f8-b440-309fd0567e1f None admin] PATH_INFO: `/v3/projects/7d41911da96e42ffa0e86c1f67b87121` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:41.319279 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240ad3ec-3499-44f8-b440-309fd0567e1f None admin] RBAC: Authorizing `identity:get_project(project_id=7d41911da96e42ffa0e86c1f67b87121)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:41.320756 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240ad3ec-3499-44f8-b440-309fd0567e1f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:41.322501 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 584/1160] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:03:41 2022] GET /identity/v3/projects/7d41911da96e42ffa0e86c1f67b87121 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:41.328892 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da6f9e-0cf3-49d0-8b3e-25891939a32a None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:41.434522 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da6f9e-0cf3-49d0-8b3e-25891939a32a None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:41.473407 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4f03cb-6ae9-464f-8617-010b9506bef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:41.474058 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f03cb-6ae9-464f-8617-010b9506bef0 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:41.474344 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f03cb-6ae9-464f-8617-010b9506bef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:41.474616 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f03cb-6ae9-464f-8617-010b9506bef0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:41.475482 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4f03cb-6ae9-464f-8617-010b9506bef0 None admin] RBAC: Adding query filter params (role.id=8cfd0b6e71e14e0983c3eb4d7478e027, scope.project.id=7d41911da96e42ffa0e86c1f67b87121, user.id=ae4c7fa8d4004382b2cd843c200af3a8) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:03:41.475822 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4f03cb-6ae9-464f-8617-010b9506bef0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:41.477273 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4f03cb-6ae9-464f-8617-010b9506bef0 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:41.480289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:41.484016 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 577/1161] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:03:41 2022] GET /identity/v3/role_assignments?user.id=ae4c7fa8d4004382b2cd843c200af3a8&scope.project.id=7d41911da96e42ffa0e86c1f67b87121&role.id=8cfd0b6e71e14e0983c3eb4d7478e027 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:43.607115 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35018bfc-9741-47ba-8198-3bd58cec27aa None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:43.607700 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35018bfc-9741-47ba-8198-3bd58cec27aa None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:43.607878 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35018bfc-9741-47ba-8198-3bd58cec27aa None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:43.609207 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 585/1162] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:03:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:03:43.615297 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9eaa2-39f4-41bd-b281-bee4ba22dc01 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:43.615804 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9eaa2-39f4-41bd-b281-bee4ba22dc01 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:43.615907 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9eaa2-39f4-41bd-b281-bee4ba22dc01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:43.642334 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-8cb9eaa2-39f4-41bd-b281-bee4ba22dc01 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:03:43.644207 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-8cb9eaa2-39f4-41bd-b281-bee4ba22dc01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:03:43.714652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 578/1163] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:03:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 23:03:43.738018 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35018bfc-9741-47ba-8198-3bd58cec27aa None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:43.913666 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35018bfc-9741-47ba-8198-3bd58cec27aa None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:43.971491 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee7ac6-c3f9-420b-8ae4-3d4a70513e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:43.972610 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee7ac6-c3f9-420b-8ae4-3d4a70513e4e None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:43.973094 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee7ac6-c3f9-420b-8ae4-3d4a70513e4e None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:43.973569 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee7ac6-c3f9-420b-8ae4-3d4a70513e4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:44.071413 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:44.089452 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee7ac6-c3f9-420b-8ae4-3d4a70513e4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:44.092126 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee7ac6-c3f9-420b-8ae4-3d4a70513e4e None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:44.258217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 586/1164] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:03:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 23:03:44.264366 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb9eaa2-39f4-41bd-b281-bee4ba22dc01 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:44.374691 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb9eaa2-39f4-41bd-b281-bee4ba22dc01 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:44.404594 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:44.411390 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1799df-c139-4c2e-9c99-67817d8087fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:44.412244 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1799df-c139-4c2e-9c99-67817d8087fb None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:44.412526 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1799df-c139-4c2e-9c99-67817d8087fb None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:44.412807 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1799df-c139-4c2e-9c99-67817d8087fb None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:44.420314 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1799df-c139-4c2e-9c99-67817d8087fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:44.421794 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1799df-c139-4c2e-9c99-67817d8087fb None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:44.437113 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-5d1799df-c139-4c2e-9c99-67817d8087fb None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 13 23:03:44.437113 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:03:44.437113 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 13 23:03:44.437113 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 13 23:03:44.437113 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 23:03:44.437113 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 23:03:44.437113 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 23:03:44.437113 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 23:03:44.437113 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 13 23:03:44.437113 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:03:44.437113 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 23:03:44.437113 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:03:44.437113 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:03:44.437113 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:03:44.437113 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:03:44.437113 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:03:44.437113 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:03:44.438379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:03:44.438379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:03:44.438379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:03:44.438379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:03:44.438379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:03:44.438379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:03:44.438379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:03:44.438379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:03:44.438379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:03:44.438379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:03:44.438379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:03:44.438379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:03:44.438379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:03:44.438379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:03:44.438379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:03:44.438379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:03:44.439400 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:03:44.439400 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:03:44.439400 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:03:44.439400 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:03:44.439400 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:03:44.439400 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:03:44.439400 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:03:44.439400 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:03:44.439400 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:03:44.439400 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:03:44.439400 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 13 23:03:44.439400 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:03:44.439400 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 13 23:03:44.439400 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:03:44.443874 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 579/1165] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Jul 13 23:03:44 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:03:44.444934 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee7ac6-c3f9-420b-8ae4-3d4a70513e4e None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:44.560345 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:44.572273 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee7ac6-c3f9-420b-8ae4-3d4a70513e4e None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:44.608820 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b62d4-e28f-4118-9be8-39bcd88ee376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:44.609503 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b62d4-e28f-4118-9be8-39bcd88ee376 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:44.609751 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b62d4-e28f-4118-9be8-39bcd88ee376 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:44.610025 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b62d4-e28f-4118-9be8-39bcd88ee376 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:44.610756 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9b62d4-e28f-4118-9be8-39bcd88ee376 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:03:44.611105 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9b62d4-e28f-4118-9be8-39bcd88ee376 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:44.613475 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9b62d4-e28f-4118-9be8-39bcd88ee376 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:44.621662 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 587/1166] 10.222.0.60 () {56 vars in 1115 bytes} [Wed Jul 13 23:03:44 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:46.714309 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec6ffc1-a4a7-4f63-9cc1-51ca6b012883 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:46.714885 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec6ffc1-a4a7-4f63-9cc1-51ca6b012883 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:46.714885 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec6ffc1-a4a7-4f63-9cc1-51ca6b012883 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:46.716303 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 580/1167] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:03:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:03:46.722779 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3818de5f-b2a5-4709-8402-91cbf258e816 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:46.723250 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3818de5f-b2a5-4709-8402-91cbf258e816 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:46.723431 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3818de5f-b2a5-4709-8402-91cbf258e816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:46.740950 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-3818de5f-b2a5-4709-8402-91cbf258e816 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:03:46.742225 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3818de5f-b2a5-4709-8402-91cbf258e816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:03:46.812263 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 588/1168] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:03:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 23:03:46.835893 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec6ffc1-a4a7-4f63-9cc1-51ca6b012883 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:46.944314 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec6ffc1-a4a7-4f63-9cc1-51ca6b012883 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:46.981207 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a8829-cfe4-43a4-8376-3fea5d2a5a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:46.981902 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a8829-cfe4-43a4-8376-3fea5d2a5a7c None admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:46.982275 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a8829-cfe4-43a4-8376-3fea5d2a5a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:46.982632 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a8829-cfe4-43a4-8376-3fea5d2a5a7c None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:46.983477 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a8829-cfe4-43a4-8376-3fea5d2a5a7c None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:46.984978 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a8829-cfe4-43a4-8376-3fea5d2a5a7c None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:47.003575 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 581/1169] 10.222.0.60 () {60 vars in 1138 bytes} [Wed Jul 13 23:03:46 2022] POST /identity/v3/domains => generated 267 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:03:49.121759 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d7514-e3c2-473a-b683-978d623ef632 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:49.122309 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d7514-e3c2-473a-b683-978d623ef632 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:49.122309 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d7514-e3c2-473a-b683-978d623ef632 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:49.123751 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 589/1170] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:03:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:03:49.131781 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a94622-18ca-4711-8d4c-0bc48d8dd256 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:49.132289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a94622-18ca-4711-8d4c-0bc48d8dd256 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:49.132688 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a94622-18ca-4711-8d4c-0bc48d8dd256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:49.158783 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-b5a94622-18ca-4711-8d4c-0bc48d8dd256 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:03:49.160467 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-b5a94622-18ca-4711-8d4c-0bc48d8dd256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:03:49.232389 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:49.259575 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 582/1171] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:03:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:03:49.283839 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2d7514-e3c2-473a-b683-978d623ef632 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:49.374933 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:49.394905 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2d7514-e3c2-473a-b683-978d623ef632 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:49.431317 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5104afe-2818-46dc-84fc-00bd4bf3e809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:49.432294 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5104afe-2818-46dc-84fc-00bd4bf3e809 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:49.432703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5104afe-2818-46dc-84fc-00bd4bf3e809 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:49.433031 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5104afe-2818-46dc-84fc-00bd4bf3e809 None admin] PATH_INFO: `/v3/domains/c7ac8c2f68e64ad6a41537a99634fef3` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:49.445744 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5104afe-2818-46dc-84fc-00bd4bf3e809 None admin] RBAC: Authorizing `identity:get_domain(domain_id=c7ac8c2f68e64ad6a41537a99634fef3)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:49.447279 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5104afe-2818-46dc-84fc-00bd4bf3e809 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:49.449137 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 590/1172] 10.222.0.60 () {56 vars in 1183 bytes} [Wed Jul 13 23:03:49 2022] GET /identity/v3/domains/c7ac8c2f68e64ad6a41537a99634fef3 => generated 267 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:49.455395 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a94622-18ca-4711-8d4c-0bc48d8dd256 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:49.558284 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a94622-18ca-4711-8d4c-0bc48d8dd256 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:49.594070 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f3cc89-76a0-4c32-ad64-45f09935360f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:49.594811 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f3cc89-76a0-4c32-ad64-45f09935360f None admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:49.595129 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f3cc89-76a0-4c32-ad64-45f09935360f None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:49.595453 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f3cc89-76a0-4c32-ad64-45f09935360f None admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:49.596820 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f3cc89-76a0-4c32-ad64-45f09935360f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:49.598394 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f3cc89-76a0-4c32-ad64-45f09935360f None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:49.628604 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 583/1173] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Jul 13 23:03:49 2022] POST /identity/v3/projects => generated 361 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:03:51.711289 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd1dc6-9f4c-4cf6-93ad-882a5e31a987 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:51.711883 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd1dc6-9f4c-4cf6-93ad-882a5e31a987 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:51.711883 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd1dc6-9f4c-4cf6-93ad-882a5e31a987 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:51.713337 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 591/1174] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:03:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:03:51.719444 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58578d-5355-40e4-b512-a525d12ff168 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:51.720054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58578d-5355-40e4-b512-a525d12ff168 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:51.720054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58578d-5355-40e4-b512-a525d12ff168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:51.739392 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-9d58578d-5355-40e4-b512-a525d12ff168 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:03:51.740792 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-9d58578d-5355-40e4-b512-a525d12ff168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:03:51.771319 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:51.820419 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 584/1175] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:03:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:03:51.846144 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd1dc6-9f4c-4cf6-93ad-882a5e31a987 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:51.975345 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd1dc6-9f4c-4cf6-93ad-882a5e31a987 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:52.010099 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ef1e5-2131-4b15-bd8b-11b3777daf4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:52.010967 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ef1e5-2131-4b15-bd8b-11b3777daf4d None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:52.011248 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ef1e5-2131-4b15-bd8b-11b3777daf4d None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:52.011521 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ef1e5-2131-4b15-bd8b-11b3777daf4d None admin] PATH_INFO: `/v3/domains/c7ac8c2f68e64ad6a41537a99634fef3` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:52.012744 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26ef1e5-2131-4b15-bd8b-11b3777daf4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=c7ac8c2f68e64ad6a41537a99634fef3)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:52.014181 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26ef1e5-2131-4b15-bd8b-11b3777daf4d None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:52.015908 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 592/1176] 10.222.0.60 () {56 vars in 1183 bytes} [Wed Jul 13 23:03:51 2022] GET /identity/v3/domains/c7ac8c2f68e64ad6a41537a99634fef3 => generated 267 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:52.024640 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58578d-5355-40e4-b512-a525d12ff168 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:52.130482 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58578d-5355-40e4-b512-a525d12ff168 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:52.165759 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc698dc5-1d52-42a6-b88e-9a394d3e8c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:52.166431 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc698dc5-1d52-42a6-b88e-9a394d3e8c8d None admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:52.166715 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc698dc5-1d52-42a6-b88e-9a394d3e8c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:52.166992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc698dc5-1d52-42a6-b88e-9a394d3e8c8d None admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:52.167746 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc698dc5-1d52-42a6-b88e-9a394d3e8c8d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:52.169208 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc698dc5-1d52-42a6-b88e-9a394d3e8c8d None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:52.191656 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 585/1177] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Jul 13 23:03:52 2022] POST /identity/v3/users => generated 311 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:03:54.312145 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c6bb0-dd4c-4a2c-9c60-3373a4cd88c9 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:54.312980 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c6bb0-dd4c-4a2c-9c60-3373a4cd88c9 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:54.312980 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c6bb0-dd4c-4a2c-9c60-3373a4cd88c9 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:54.314273 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 593/1178] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:03:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:03:54.320396 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec1933-f227-4d66-a67c-bed96c68bdbc None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:54.321166 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec1933-f227-4d66-a67c-bed96c68bdbc None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:54.321319 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec1933-f227-4d66-a67c-bed96c68bdbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:54.338795 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-fbec1933-f227-4d66-a67c-bed96c68bdbc None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:03:54.340043 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-fbec1933-f227-4d66-a67c-bed96c68bdbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:03:54.354934 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:54.412784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 586/1179] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:03:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 23:03:54.436234 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67c6bb0-dd4c-4a2c-9c60-3373a4cd88c9 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:54.452574 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:54.544335 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67c6bb0-dd4c-4a2c-9c60-3373a4cd88c9 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:54.580564 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31ee4a-f58e-445e-aac5-2e7934ca5809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:54.581270 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31ee4a-f58e-445e-aac5-2e7934ca5809 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:54.581544 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31ee4a-f58e-445e-aac5-2e7934ca5809 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:54.581809 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31ee4a-f58e-445e-aac5-2e7934ca5809 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:54.593370 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a31ee4a-f58e-445e-aac5-2e7934ca5809 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:03:54.601613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-3a31ee4a-f58e-445e-aac5-2e7934ca5809 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:03:54.601613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:03:54.601613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:03:54.601613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:03:54.601613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:03:54.601613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:03:54.601613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:03:54.601613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:03:54.601613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:03:54.601613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:03:54.601613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:03:54.601613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:03:54.601613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:03:54.601613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:03:54.601613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:03:54.601613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:03:54.602736 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:03:54.602736 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:03:54.602736 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:03:54.602736 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:03:54.602736 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:03:54.602736 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:03:54.602736 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:03:54.602736 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:03:54.602736 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:03:54.602736 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:03:54.602736 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:03:54.602736 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:03:54.602736 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:03:54.602736 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:03:54.602736 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:03:54.602736 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:03:54.603647 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:03:54.603647 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:03:54.603647 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:03:54.603647 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:03:54.603647 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:03:54.603647 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:03:54.603647 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:03:54.603647 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:03:54.603647 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:03:54.603647 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:03:54.603647 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:03:54.603647 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:03:54.603647 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 594/1180] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jul 13 23:03:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:03:54.609773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec1933-f227-4d66-a67c-bed96c68bdbc None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:54.740457 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec1933-f227-4d66-a67c-bed96c68bdbc None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:54.775397 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cb4462-b059-4a47-bd11-b0d0c299537d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:54.776103 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb4462-b059-4a47-bd11-b0d0c299537d None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:54.776384 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb4462-b059-4a47-bd11-b0d0c299537d None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:54.776661 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb4462-b059-4a47-bd11-b0d0c299537d None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:54.777430 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cb4462-b059-4a47-bd11-b0d0c299537d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:03:54.777739 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cb4462-b059-4a47-bd11-b0d0c299537d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:54.779146 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cb4462-b059-4a47-bd11-b0d0c299537d None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:54.788485 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 587/1181] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:03:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:54.794568 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31ee4a-f58e-445e-aac5-2e7934ca5809 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:54.871636 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:54.902386 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31ee4a-f58e-445e-aac5-2e7934ca5809 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:54.936696 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dce204-8d52-46a9-8bac-c6b8b26b1bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:54.937560 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dce204-8d52-46a9-8bac-c6b8b26b1bf1 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:54.937836 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dce204-8d52-46a9-8bac-c6b8b26b1bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:54.938112 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dce204-8d52-46a9-8bac-c6b8b26b1bf1 None admin] PATH_INFO: `/v3/users/77b5d985db924237af18861ed6af4545` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:54.953207 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dce204-8d52-46a9-8bac-c6b8b26b1bf1 None admin] RBAC: Authorizing `identity:get_user(user_id=77b5d985db924237af18861ed6af4545)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:54.955222 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dce204-8d52-46a9-8bac-c6b8b26b1bf1 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:54.957026 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 595/1182] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:03:54 2022] GET /identity/v3/users/77b5d985db924237af18861ed6af4545 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:54.963128 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cb4462-b059-4a47-bd11-b0d0c299537d None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:54.977724 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:55.083386 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cb4462-b059-4a47-bd11-b0d0c299537d None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:55.122059 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6059b7a4-90d7-4b1d-95d6-135549f5aa90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:55.122701 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059b7a4-90d7-4b1d-95d6-135549f5aa90 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:55.123060 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059b7a4-90d7-4b1d-95d6-135549f5aa90 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:55.123406 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059b7a4-90d7-4b1d-95d6-135549f5aa90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:55.199514 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6059b7a4-90d7-4b1d-95d6-135549f5aa90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:55.201026 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6059b7a4-90d7-4b1d-95d6-135549f5aa90 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:55.213268 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:55.324538 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 588/1183] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:03:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 23:03:55.331222 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dce204-8d52-46a9-8bac-c6b8b26b1bf1 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:55.452180 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dce204-8d52-46a9-8bac-c6b8b26b1bf1 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:55.489089 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:55.496862 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df8993-909f-4eab-9250-dcd51a046e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:55.497823 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df8993-909f-4eab-9250-dcd51a046e27 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:55.498152 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df8993-909f-4eab-9250-dcd51a046e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:55.498476 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df8993-909f-4eab-9250-dcd51a046e27 None admin] PATH_INFO: `/v3/projects/88a08d5c7c784d839fa218a9288ca6ae` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:55.500069 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df8993-909f-4eab-9250-dcd51a046e27 None admin] RBAC: Authorizing `identity:get_project(project_id=88a08d5c7c784d839fa218a9288ca6ae)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:55.501629 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df8993-909f-4eab-9250-dcd51a046e27 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:55.503630 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 596/1184] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:03:55 2022] GET /identity/v3/projects/88a08d5c7c784d839fa218a9288ca6ae => generated 361 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:55.510280 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6059b7a4-90d7-4b1d-95d6-135549f5aa90 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:55.620768 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6059b7a4-90d7-4b1d-95d6-135549f5aa90 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:55.633006 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:55.659407 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f082386-0c27-42fe-a6fa-c00771b7b2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:55.660098 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f082386-0c27-42fe-a6fa-c00771b7b2e3 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:55.660378 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f082386-0c27-42fe-a6fa-c00771b7b2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:55.660649 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f082386-0c27-42fe-a6fa-c00771b7b2e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:55.661487 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f082386-0c27-42fe-a6fa-c00771b7b2e3 None admin] RBAC: Adding query filter params (role.id=8cfd0b6e71e14e0983c3eb4d7478e027, scope.project.id=88a08d5c7c784d839fa218a9288ca6ae, user.id=77b5d985db924237af18861ed6af4545) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:03:55.661794 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f082386-0c27-42fe-a6fa-c00771b7b2e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:55.663245 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f082386-0c27-42fe-a6fa-c00771b7b2e3 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:55.669713 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 589/1185] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:03:55 2022] GET /identity/v3/role_assignments?user.id=77b5d985db924237af18861ed6af4545&scope.project.id=88a08d5c7c784d839fa218a9288ca6ae&role.id=8cfd0b6e71e14e0983c3eb4d7478e027 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:03:57.755720 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dcc21a-427b-4459-95ab-bb8feb54dfce None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:57.756476 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dcc21a-427b-4459-95ab-bb8feb54dfce None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:57.756476 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dcc21a-427b-4459-95ab-bb8feb54dfce None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:57.757655 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 597/1186] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:03:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:03:57.763382 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaeafb7-975d-40b5-8601-fee47773f47b None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:57.763954 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaeafb7-975d-40b5-8601-fee47773f47b None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:57.764087 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaeafb7-975d-40b5-8601-fee47773f47b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:57.782840 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-bfaeafb7-975d-40b5-8601-fee47773f47b None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:03:57.784086 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-bfaeafb7-975d-40b5-8601-fee47773f47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:03:57.854612 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 590/1187] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:03:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 23:03:57.878517 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dcc21a-427b-4459-95ab-bb8feb54dfce None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:57.988897 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dcc21a-427b-4459-95ab-bb8feb54dfce None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:58.025273 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7c6c5-0305-4724-8228-7b6219f6a773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:58.025977 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7c6c5-0305-4724-8228-7b6219f6a773 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:58.026258 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7c6c5-0305-4724-8228-7b6219f6a773 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:58.026532 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7c6c5-0305-4724-8228-7b6219f6a773 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:58.037573 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda7c6c5-0305-4724-8228-7b6219f6a773 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:03:58.045298 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-dda7c6c5-0305-4724-8228-7b6219f6a773 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:03:58.045298 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:03:58.045298 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:03:58.045298 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:03:58.045298 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:03:58.045298 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:03:58.045298 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:03:58.045298 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:03:58.045298 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:03:58.045298 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:03:58.045298 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:03:58.045298 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:03:58.045298 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:03:58.045298 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:03:58.045298 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:03:58.045298 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:03:58.047206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:03:58.047206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:03:58.047206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:03:58.047206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:03:58.047206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:03:58.047206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:03:58.047206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:03:58.047206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:03:58.047206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:03:58.047206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:03:58.047206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:03:58.047206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:03:58.047206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:03:58.047206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:03:58.047206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:03:58.047206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:03:58.049364 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:03:58.049364 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:03:58.049364 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:03:58.049364 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:03:58.049364 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:03:58.049364 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:03:58.049364 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:03:58.049364 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:03:58.049364 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:03:58.049364 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:03:58.049364 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:03:58.049364 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:03:58.049364 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 598/1188] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jul 13 23:03:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:03:58.054601 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaeafb7-975d-40b5-8601-fee47773f47b None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:58.175228 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaeafb7-975d-40b5-8601-fee47773f47b None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:58.178005 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:58.212996 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ee16b-1813-44cc-b66c-90fbdbf0e2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:58.213667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ee16b-1813-44cc-b66c-90fbdbf0e2e5 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:58.213951 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ee16b-1813-44cc-b66c-90fbdbf0e2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:58.214228 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ee16b-1813-44cc-b66c-90fbdbf0e2e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:58.214989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ee16b-1813-44cc-b66c-90fbdbf0e2e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:03:58.215293 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ee16b-1813-44cc-b66c-90fbdbf0e2e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:58.216735 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ee16b-1813-44cc-b66c-90fbdbf0e2e5 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:58.225777 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 591/1189] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:03:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:03:58.231814 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7c6c5-0305-4724-8228-7b6219f6a773 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:58.296446 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:58.343310 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7c6c5-0305-4724-8228-7b6219f6a773 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:58.383605 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a9dbfb-5fe8-47af-aefa-0eb12220cee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:58.384493 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9dbfb-5fe8-47af-aefa-0eb12220cee9 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:58.384773 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9dbfb-5fe8-47af-aefa-0eb12220cee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:58.385047 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9dbfb-5fe8-47af-aefa-0eb12220cee9 None admin] PATH_INFO: `/v3/users/77b5d985db924237af18861ed6af4545` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:58.386249 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a9dbfb-5fe8-47af-aefa-0eb12220cee9 None admin] RBAC: Authorizing `identity:get_user(user_id=77b5d985db924237af18861ed6af4545)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:58.387773 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a9dbfb-5fe8-47af-aefa-0eb12220cee9 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:58.389469 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 599/1190] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:03:58 2022] GET /identity/v3/users/77b5d985db924237af18861ed6af4545 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:03:58.395947 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ee16b-1813-44cc-b66c-90fbdbf0e2e5 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:58.525250 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ee16b-1813-44cc-b66c-90fbdbf0e2e5 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:58.586671 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dcc42b-2614-4b0c-819a-a8ff9b96e9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:58.588147 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dcc42b-2614-4b0c-819a-a8ff9b96e9e9 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:58.588663 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dcc42b-2614-4b0c-819a-a8ff9b96e9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:58.589171 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dcc42b-2614-4b0c-819a-a8ff9b96e9e9 None admin] PATH_INFO: `/v3/projects/88a08d5c7c784d839fa218a9288ca6ae` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:58.591350 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dcc42b-2614-4b0c-819a-a8ff9b96e9e9 None admin] RBAC: Authorizing `identity:get_project(project_id=88a08d5c7c784d839fa218a9288ca6ae)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:58.594176 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dcc42b-2614-4b0c-819a-a8ff9b96e9e9 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:58.597719 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 592/1191] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:03:58 2022] GET /identity/v3/projects/88a08d5c7c784d839fa218a9288ca6ae => generated 361 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:03:58.603842 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a9dbfb-5fe8-47af-aefa-0eb12220cee9 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:03:58.711815 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a9dbfb-5fe8-47af-aefa-0eb12220cee9 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:03:58.738179 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:03:58.748000 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8415b6-fc2d-48b3-9651-73a03730e562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:03:58.748897 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8415b6-fc2d-48b3-9651-73a03730e562 None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:03:58.749178 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8415b6-fc2d-48b3-9651-73a03730e562 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:03:58.749452 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8415b6-fc2d-48b3-9651-73a03730e562 None admin] PATH_INFO: `/v3/projects/88a08d5c7c784d839fa218a9288ca6ae/users/77b5d985db924237af18861ed6af4545/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:03:58.752005 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8415b6-fc2d-48b3-9651-73a03730e562 None admin] RBAC: Authorizing `identity:create_grant(project_id=88a08d5c7c784d839fa218a9288ca6ae, user_id=77b5d985db924237af18861ed6af4545, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:03:58.753430 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8415b6-fc2d-48b3-9651-73a03730e562 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:03:58.767662 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 600/1192] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jul 13 23:03:58 2022] PUT /identity/v3/projects/88a08d5c7c784d839fa218a9288ca6ae/users/77b5d985db924237af18861ed6af4545/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:04:00.902273 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35034de1-54c0-4079-920f-683106083ed5 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:04:00.902829 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35034de1-54c0-4079-920f-683106083ed5 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:04:00.902950 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35034de1-54c0-4079-920f-683106083ed5 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:04:00.904402 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 593/1193] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:04:00.910681 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06508c78-fcbf-424c-8a82-e06e001d933a None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:04:00.911203 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06508c78-fcbf-424c-8a82-e06e001d933a None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:04:00.911326 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06508c78-fcbf-424c-8a82-e06e001d933a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:04:00.930690 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-06508c78-fcbf-424c-8a82-e06e001d933a None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:04:00.932016 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-06508c78-fcbf-424c-8a82-e06e001d933a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:04:01.007431 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 601/1194] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:04:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:04:01.032243 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35034de1-54c0-4079-920f-683106083ed5 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:04:01.072182 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:04:01.145719 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35034de1-54c0-4079-920f-683106083ed5 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:04:01.192763 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d25059-37fd-4309-bbb0-e28a64d5673f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:04:01.193530 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d25059-37fd-4309-bbb0-e28a64d5673f None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:04:01.193851 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d25059-37fd-4309-bbb0-e28a64d5673f None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:04:01.194169 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d25059-37fd-4309-bbb0-e28a64d5673f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:04:01.205546 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d25059-37fd-4309-bbb0-e28a64d5673f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:04:01.213848 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-25d25059-37fd-4309-bbb0-e28a64d5673f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:04:01.213848 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:04:01.213848 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:04:01.213848 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:04:01.213848 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:04:01.213848 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:04:01.213848 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:04:01.213848 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:04:01.213848 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:04:01.213848 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:04:01.213848 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:04:01.213848 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:04:01.213848 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:04:01.213848 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:04:01.213848 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:04:01.213848 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:04:01.215699 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:04:01.215699 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:04:01.215699 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:04:01.215699 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:04:01.215699 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:04:01.215699 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:04:01.215699 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:04:01.215699 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:04:01.215699 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:04:01.215699 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:04:01.215699 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:04:01.215699 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:04:01.215699 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:04:01.215699 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:04:01.215699 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:04:01.215699 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:04:01.217620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:04:01.217620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:04:01.217620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:04:01.217620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:04:01.217620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:04:01.217620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:04:01.217620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:04:01.217620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:04:01.217620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:04:01.217620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:04:01.217620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:04:01.217620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:04:01.217620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 594/1195] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jul 13 23:04:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:04:01.223596 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06508c78-fcbf-424c-8a82-e06e001d933a None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:04:01.332998 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06508c78-fcbf-424c-8a82-e06e001d933a None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:04:01.359995 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:04:01.381374 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb6d1c-cc81-4769-9cab-994e67bbb026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:04:01.382064 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb6d1c-cc81-4769-9cab-994e67bbb026 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:04:01.382349 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb6d1c-cc81-4769-9cab-994e67bbb026 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:04:01.382621 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb6d1c-cc81-4769-9cab-994e67bbb026 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:04:01.383385 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb6d1c-cc81-4769-9cab-994e67bbb026 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:04:01.383725 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb6d1c-cc81-4769-9cab-994e67bbb026 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:04:01.385119 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb6d1c-cc81-4769-9cab-994e67bbb026 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:04:01.395934 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 602/1196] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:04:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:04:01.402021 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d25059-37fd-4309-bbb0-e28a64d5673f None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:04:01.563385 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:04:01.572051 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d25059-37fd-4309-bbb0-e28a64d5673f None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:04:01.609303 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be207f7a-9517-41f2-a3a5-4b03b78319a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:04:01.610292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be207f7a-9517-41f2-a3a5-4b03b78319a0 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:04:01.611239 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be207f7a-9517-41f2-a3a5-4b03b78319a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:04:01.611588 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be207f7a-9517-41f2-a3a5-4b03b78319a0 None admin] PATH_INFO: `/v3/users/77b5d985db924237af18861ed6af4545` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:04:01.613050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be207f7a-9517-41f2-a3a5-4b03b78319a0 None admin] RBAC: Authorizing `identity:get_user(user_id=77b5d985db924237af18861ed6af4545)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:04:01.614599 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be207f7a-9517-41f2-a3a5-4b03b78319a0 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:04:01.616444 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 595/1197] 10.222.0.60 () {56 vars in 1177 bytes} [Wed Jul 13 23:04:01 2022] GET /identity/v3/users/77b5d985db924237af18861ed6af4545 => generated 311 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:04:01.622558 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb6d1c-cc81-4769-9cab-994e67bbb026 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:04:01.741684 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb6d1c-cc81-4769-9cab-994e67bbb026 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:04:01.787084 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f8d54-ee2e-46d3-a45f-d6feff85f18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:04:01.787849 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f8d54-ee2e-46d3-a45f-d6feff85f18b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:04:01.788183 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f8d54-ee2e-46d3-a45f-d6feff85f18b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:04:01.788508 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f8d54-ee2e-46d3-a45f-d6feff85f18b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:04:01.842296 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:04:01.870434 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4f8d54-ee2e-46d3-a45f-d6feff85f18b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:04:01.871950 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4f8d54-ee2e-46d3-a45f-d6feff85f18b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:04:02.033250 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 603/1198] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:04:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 23:04:02.039219 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be207f7a-9517-41f2-a3a5-4b03b78319a0 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:04:02.146573 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be207f7a-9517-41f2-a3a5-4b03b78319a0 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:04:02.183098 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0b202-787f-40d2-a971-08aca467cc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:04:02.184002 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0b202-787f-40d2-a971-08aca467cc74 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:04:02.184296 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0b202-787f-40d2-a971-08aca467cc74 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:04:02.184573 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0b202-787f-40d2-a971-08aca467cc74 None admin] PATH_INFO: `/v3/projects/88a08d5c7c784d839fa218a9288ca6ae` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:04:02.185803 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f0b202-787f-40d2-a971-08aca467cc74 None admin] RBAC: Authorizing `identity:get_project(project_id=88a08d5c7c784d839fa218a9288ca6ae)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:04:02.187259 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f0b202-787f-40d2-a971-08aca467cc74 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:04:02.189119 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 596/1199] 10.222.0.60 () {56 vars in 1186 bytes} [Wed Jul 13 23:04:02 2022] GET /identity/v3/projects/88a08d5c7c784d839fa218a9288ca6ae => generated 361 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:04:02.195749 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f8d54-ee2e-46d3-a45f-d6feff85f18b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:04:02.281694 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:04:02.304137 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f8d54-ee2e-46d3-a45f-d6feff85f18b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:04:02.341582 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a4acc5-59b7-4b34-be92-fbe004b83bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:04:02.342239 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a4acc5-59b7-4b34-be92-fbe004b83bbe None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:04:02.342519 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a4acc5-59b7-4b34-be92-fbe004b83bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:04:02.342794 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a4acc5-59b7-4b34-be92-fbe004b83bbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:04:02.343659 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a4acc5-59b7-4b34-be92-fbe004b83bbe None admin] RBAC: Adding query filter params (role.id=8cfd0b6e71e14e0983c3eb4d7478e027, scope.project.id=88a08d5c7c784d839fa218a9288ca6ae, user.id=77b5d985db924237af18861ed6af4545) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:04:02.343971 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a4acc5-59b7-4b34-be92-fbe004b83bbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:04:02.345462 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a4acc5-59b7-4b34-be92-fbe004b83bbe None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:04:02.351896 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 604/1200] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:04:02 2022] GET /identity/v3/role_assignments?user.id=77b5d985db924237af18861ed6af4545&scope.project.id=88a08d5c7c784d839fa218a9288ca6ae&role.id=8cfd0b6e71e14e0983c3eb4d7478e027 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:04:12.178863 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=85 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:04:12.348960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=29 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:00.122002 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec7bff-6c15-4acf-af3a-d0594e407000 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:00.122792 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec7bff-6c15-4acf-af3a-d0594e407000 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:00.122792 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec7bff-6c15-4acf-af3a-d0594e407000 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:00.124077 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 597/1201] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:05:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:05:00.129912 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72c0cb-dc6b-45ed-849a-b745cc01d81c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:00.130408 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72c0cb-dc6b-45ed-849a-b745cc01d81c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:00.130514 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72c0cb-dc6b-45ed-849a-b745cc01d81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:00.148643 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-6c72c0cb-dc6b-45ed-849a-b745cc01d81c None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:05:00.150213 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6c72c0cb-dc6b-45ed-849a-b745cc01d81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:05:00.235148 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 605/1202] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:05:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:05:00.260382 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec7bff-6c15-4acf-af3a-d0594e407000 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:00.412337 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec7bff-6c15-4acf-af3a-d0594e407000 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:00.456369 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1409846-6ac0-4049-beaa-39ed3fa9fd78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:00.457175 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1409846-6ac0-4049-beaa-39ed3fa9fd78 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:00.457460 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1409846-6ac0-4049-beaa-39ed3fa9fd78 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:00.457752 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1409846-6ac0-4049-beaa-39ed3fa9fd78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:00.470307 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1409846-6ac0-4049-beaa-39ed3fa9fd78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:00.471874 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1409846-6ac0-4049-beaa-39ed3fa9fd78 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:00.495670 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-a1409846-6ac0-4049-beaa-39ed3fa9fd78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:05:00.495670 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:05:00.495670 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:05:00.495670 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:05:00.495670 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:05:00.495670 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:05:00.495670 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:05:00.495670 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:05:00.495670 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:05:00.495670 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:05:00.495670 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:05:00.495670 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:05:00.495670 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:05:00.495670 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:05:00.495670 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:05:00.496886 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:05:00.496886 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:05:00.496886 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:05:00.496886 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:05:00.496886 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:05:00.496886 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:05:00.496886 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:05:00.496886 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:05:00.496886 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:05:00.496886 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:05:00.496886 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:05:00.496886 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:05:00.496886 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:05:00.496886 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:05:00.496886 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:05:00.496886 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:05:00.496886 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:05:00.497938 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:05:00.497938 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:05:00.497938 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:05:00.497938 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:05:00.497938 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 598/1203] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:05:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:05:00.503445 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c72c0cb-dc6b-45ed-849a-b745cc01d81c None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:00.632071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c72c0cb-dc6b-45ed-849a-b745cc01d81c None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:00.683522 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:00.689624 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91fd2dd-0888-4283-bac1-c3dec09b6077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:00.690555 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91fd2dd-0888-4283-bac1-c3dec09b6077 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:00.690842 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91fd2dd-0888-4283-bac1-c3dec09b6077 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:00.691112 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91fd2dd-0888-4283-bac1-c3dec09b6077 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:00.692039 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91fd2dd-0888-4283-bac1-c3dec09b6077 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:00.692421 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91fd2dd-0888-4283-bac1-c3dec09b6077 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:00.693982 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91fd2dd-0888-4283-bac1-c3dec09b6077 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:00.708149 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 606/1204] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:05:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:05:00.714039 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1409846-6ac0-4049-beaa-39ed3fa9fd78 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:00.796325 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:00.859150 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1409846-6ac0-4049-beaa-39ed3fa9fd78 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:00.907046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f45de-29c6-4e9d-b553-8281ba4ef6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:00.907985 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f45de-29c6-4e9d-b553-8281ba4ef6d7 None admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:00.908293 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f45de-29c6-4e9d-b553-8281ba4ef6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:00.908576 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f45de-29c6-4e9d-b553-8281ba4ef6d7 None admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:00.912215 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f45de-29c6-4e9d-b553-8281ba4ef6d7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:00.914074 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f45de-29c6-4e9d-b553-8281ba4ef6d7 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:00.942159 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 599/1205] 10.222.0.60 () {60 vars in 1132 bytes} [Wed Jul 13 23:05:00 2022] POST /identity/v3/users => generated 251 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:05:03.269388 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83fa66-1f14-40dd-a017-bd6854d70bcf None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:03.270041 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83fa66-1f14-40dd-a017-bd6854d70bcf None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:03.270041 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83fa66-1f14-40dd-a017-bd6854d70bcf None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:03.271587 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 607/1206] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:05:03.278752 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb9589-65a2-4f1d-ab2d-df42afcb7dd5 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:03.278752 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb9589-65a2-4f1d-ab2d-df42afcb7dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:03.278752 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb9589-65a2-4f1d-ab2d-df42afcb7dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:03.296083 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-f9fb9589-65a2-4f1d-ab2d-df42afcb7dd5 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:05:03.297421 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-f9fb9589-65a2-4f1d-ab2d-df42afcb7dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:05:03.377027 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 600/1207] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:05:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:05:03.402301 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c83fa66-1f14-40dd-a017-bd6854d70bcf None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:03.517664 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c83fa66-1f14-40dd-a017-bd6854d70bcf None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:03.557989 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0273f47-3dd5-4767-a5e9-1f3c29317e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:03.561653 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0273f47-3dd5-4767-a5e9-1f3c29317e7f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:03.561653 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0273f47-3dd5-4767-a5e9-1f3c29317e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:03.561653 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0273f47-3dd5-4767-a5e9-1f3c29317e7f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:03.572795 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0273f47-3dd5-4767-a5e9-1f3c29317e7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:05:03.581709 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-f0273f47-3dd5-4767-a5e9-1f3c29317e7f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:05:03.581709 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:05:03.581709 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:05:03.581709 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:05:03.581709 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:05:03.581709 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:05:03.581709 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:05:03.581709 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:05:03.581709 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:05:03.581709 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:05:03.581709 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:05:03.581709 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:05:03.581709 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:05:03.581709 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:05:03.581709 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:05:03.581709 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:05:03.583153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:05:03.583153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:05:03.583153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:05:03.583153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:05:03.583153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:05:03.583153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:05:03.583153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:05:03.583153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:05:03.583153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:05:03.583153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:05:03.583153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:05:03.583153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:05:03.583153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:05:03.583153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:05:03.583153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:05:03.583153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:05:03.584447 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:05:03.584447 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:05:03.584447 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:05:03.584447 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:05:03.584447 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:05:03.584447 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:05:03.584447 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:05:03.584447 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:05:03.584447 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:05:03.584447 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:05:03.584447 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:05:03.584447 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:05:03.589326 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb9589-65a2-4f1d-ab2d-df42afcb7dd5 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:03.591978 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 608/1208] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jul 13 23:05:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:05:03.643629 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:03.715470 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb9589-65a2-4f1d-ab2d-df42afcb7dd5 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:03.754789 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb5b01-988a-4ffb-a23b-4192a8473a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:03.755471 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb5b01-988a-4ffb-a23b-4192a8473a4d None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:03.755774 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb5b01-988a-4ffb-a23b-4192a8473a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:03.756041 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb5b01-988a-4ffb-a23b-4192a8473a4d None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:03.756800 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb5b01-988a-4ffb-a23b-4192a8473a4d None admin] RBAC: Adding query filter params (name=service) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:03.757097 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb5b01-988a-4ffb-a23b-4192a8473a4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:03.758486 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb5b01-988a-4ffb-a23b-4192a8473a4d None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:03.768084 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 601/1209] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jul 13 23:05:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:05:03.774556 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0273f47-3dd5-4767-a5e9-1f3c29317e7f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:03.847990 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:03.910883 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0273f47-3dd5-4767-a5e9-1f3c29317e7f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:03.963716 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00789a42-6ea3-4631-b1bf-6749ce8f769f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:03.964686 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00789a42-6ea3-4631-b1bf-6749ce8f769f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:03.965068 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00789a42-6ea3-4631-b1bf-6749ce8f769f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:03.965449 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00789a42-6ea3-4631-b1bf-6749ce8f769f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:03.982294 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00789a42-6ea3-4631-b1bf-6749ce8f769f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:03.983731 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00789a42-6ea3-4631-b1bf-6749ce8f769f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:04.004501 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-00789a42-6ea3-4631-b1bf-6749ce8f769f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:05:04.004501 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:05:04.004501 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:05:04.004501 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:05:04.004501 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:05:04.004501 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:05:04.004501 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:05:04.004501 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:05:04.004501 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:05:04.004501 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:05:04.004501 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:05:04.004501 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:05:04.004501 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:05:04.004501 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:05:04.004501 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:05:04.006323 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:05:04.006323 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:05:04.006323 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:05:04.006323 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:05:04.006323 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:05:04.006323 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:05:04.006323 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:05:04.006323 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:05:04.006323 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:05:04.006323 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:05:04.006323 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:05:04.006323 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:05:04.006323 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:05:04.006323 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:05:04.006323 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:05:04.006323 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:05:04.006323 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:05:04.007464 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:05:04.007464 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:05:04.007464 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:05:04.007464 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:05:04.007464 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 609/1210] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:05:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:05:04.011788 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb5b01-988a-4ffb-a23b-4192a8473a4d None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:04.171071 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:04.174366 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb5b01-988a-4ffb-a23b-4192a8473a4d None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:04.220399 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc5d759-3fbf-426a-8602-6ce69aaa3801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:04.221066 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc5d759-3fbf-426a-8602-6ce69aaa3801 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:04.226715 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc5d759-3fbf-426a-8602-6ce69aaa3801 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:04.226715 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc5d759-3fbf-426a-8602-6ce69aaa3801 None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:04.226715 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc5d759-3fbf-426a-8602-6ce69aaa3801 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:04.226715 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc5d759-3fbf-426a-8602-6ce69aaa3801 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:04.226715 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc5d759-3fbf-426a-8602-6ce69aaa3801 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:04.242326 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 602/1211] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:05:04.249296 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00789a42-6ea3-4631-b1bf-6749ce8f769f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:04.372271 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00789a42-6ea3-4631-b1bf-6749ce8f769f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:04.374564 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:04.425588 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a23e93a-abad-4261-8738-ddf2aa273e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:04.426701 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a23e93a-abad-4261-8738-ddf2aa273e3c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:04.427212 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a23e93a-abad-4261-8738-ddf2aa273e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:04.427777 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a23e93a-abad-4261-8738-ddf2aa273e3c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:04.440280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a23e93a-abad-4261-8738-ddf2aa273e3c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:04.443019 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a23e93a-abad-4261-8738-ddf2aa273e3c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:04.462002 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-9a23e93a-abad-4261-8738-ddf2aa273e3c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 13 23:05:04.462002 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:05:04.462002 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:05:04.462002 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:05:04.462002 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:05:04.462002 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:05:04.462002 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:05:04.462002 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:05:04.462002 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:05:04.462002 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:05:04.462002 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:05:04.462002 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:05:04.462002 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 23:05:04.462002 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 23:05:04.462002 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 23:05:04.462002 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 23:05:04.463197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:05:04.463197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:05:04.463197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 23:05:04.463197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:05:04.463197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 23:05:04.463197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:05:04.463197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:05:04.463197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:05:04.463197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:05:04.463197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:05:04.463197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:05:04.463197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:05:04.463197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:05:04.463197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:05:04.463197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:05:04.463197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:05:04.463197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:05:04.463197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:05:04.464207 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:05:04.464207 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:05:04.464207 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 23:05:04.464207 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 23:05:04.464207 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 23:05:04.464207 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 23:05:04.464207 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 23:05:04.464207 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 23:05:04.464207 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 13 23:05:04.464207 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:05:04.467681 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 610/1212] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:05:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:05:04.469193 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc5d759-3fbf-426a-8602-6ce69aaa3801 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:04.603700 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc5d759-3fbf-426a-8602-6ce69aaa3801 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:04.640914 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec00d0-4441-4e30-9386-3066cf3f3a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:04.641807 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec00d0-4441-4e30-9386-3066cf3f3a61 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:04.642153 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec00d0-4441-4e30-9386-3066cf3f3a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:04.642548 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec00d0-4441-4e30-9386-3066cf3f3a61 None admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:04.643502 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ec00d0-4441-4e30-9386-3066cf3f3a61 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:04.643901 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ec00d0-4441-4e30-9386-3066cf3f3a61 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:04.645570 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ec00d0-4441-4e30-9386-3066cf3f3a61 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:04.671567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 603/1213] 10.222.0.60 () {56 vars in 1143 bytes} [Wed Jul 13 23:05:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:05:04.672862 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a23e93a-abad-4261-8738-ddf2aa273e3c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:04.798677 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a23e93a-abad-4261-8738-ddf2aa273e3c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:04.848089 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6055335-b07e-4d63-9b64-8e4aea21871c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:04.848765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6055335-b07e-4d63-9b64-8e4aea21871c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:04.849125 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6055335-b07e-4d63-9b64-8e4aea21871c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:04.849405 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6055335-b07e-4d63-9b64-8e4aea21871c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:04.888724 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:04.934785 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6055335-b07e-4d63-9b64-8e4aea21871c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:04.936932 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6055335-b07e-4d63-9b64-8e4aea21871c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:05.075730 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 611/1214] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:05:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 23:05:05.082040 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec00d0-4441-4e30-9386-3066cf3f3a61 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:05.114586 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:05.249998 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec00d0-4441-4e30-9386-3066cf3f3a61 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:05.300378 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383287c4-31cb-4add-b6a6-533578c264d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:05.301751 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383287c4-31cb-4add-b6a6-533578c264d3 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:05.302184 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383287c4-31cb-4add-b6a6-533578c264d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:05.302639 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383287c4-31cb-4add-b6a6-533578c264d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:05.316461 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383287c4-31cb-4add-b6a6-533578c264d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:05.318175 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383287c4-31cb-4add-b6a6-533578c264d3 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:05.338167 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-383287c4-31cb-4add-b6a6-533578c264d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:05:05.338167 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:05:05.338167 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:05:05.338167 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:05:05.338167 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:05:05.338167 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:05:05.338167 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:05:05.338167 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:05:05.338167 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:05:05.338167 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:05:05.338167 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:05:05.338167 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:05:05.338167 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:05:05.338167 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:05:05.338167 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:05:05.340144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:05:05.340144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:05:05.340144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:05:05.340144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:05:05.340144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:05:05.340144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:05:05.340144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:05:05.340144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:05:05.340144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:05:05.340144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:05:05.340144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:05:05.340144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:05:05.340144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:05:05.340144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:05:05.340144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:05:05.340144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:05:05.340144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:05:05.341972 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:05:05.341972 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:05:05.341972 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:05:05.341972 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:05:05.341972 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 604/1215] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:05:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:05:05.346672 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6055335-b07e-4d63-9b64-8e4aea21871c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:05.416738 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:05.470635 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6055335-b07e-4d63-9b64-8e4aea21871c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:05.510779 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3328c4-bdd0-4c32-94b9-6278a962e86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:05.511455 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3328c4-bdd0-4c32-94b9-6278a962e86c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:05.511800 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3328c4-bdd0-4c32-94b9-6278a962e86c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:05.512036 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3328c4-bdd0-4c32-94b9-6278a962e86c None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:05.512751 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3328c4-bdd0-4c32-94b9-6278a962e86c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:05.513059 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3328c4-bdd0-4c32-94b9-6278a962e86c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:05.514470 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3328c4-bdd0-4c32-94b9-6278a962e86c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:05.527264 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 612/1216] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:05:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:05:05.536619 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383287c4-31cb-4add-b6a6-533578c264d3 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:05.617647 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:05.655579 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383287c4-31cb-4add-b6a6-533578c264d3 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:05.695221 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea45e3-fbef-411e-8d4c-a6de54e8b1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:05.696172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea45e3-fbef-411e-8d4c-a6de54e8b1d2 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:05.696454 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea45e3-fbef-411e-8d4c-a6de54e8b1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:05.696733 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea45e3-fbef-411e-8d4c-a6de54e8b1d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:05.703972 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ea45e3-fbef-411e-8d4c-a6de54e8b1d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:05.705623 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ea45e3-fbef-411e-8d4c-a6de54e8b1d2 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:05.720493 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-a0ea45e3-fbef-411e-8d4c-a6de54e8b1d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:05:05.720493 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:05:05.720493 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:05:05.720493 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:05:05.720493 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:05:05.720493 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:05:05.720493 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:05:05.720493 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:05:05.720493 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:05:05.720493 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:05:05.720493 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:05:05.720493 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:05:05.720493 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 23:05:05.720493 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 23:05:05.720493 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 23:05:05.721490 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 23:05:05.721490 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:05:05.721490 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:05:05.721490 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:05:05.721490 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:05:05.721490 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:05:05.721490 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:05:05.721490 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:05:05.721490 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:05:05.721490 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:05:05.721490 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:05:05.721490 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:05:05.721490 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:05:05.721490 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:05:05.721490 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:05:05.721490 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:05:05.721490 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:05:05.722400 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 23:05:05.722400 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 23:05:05.722400 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 23:05:05.722400 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 23:05:05.722400 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 23:05:05.722400 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 23:05:05.722400 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:05:05.722400 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:05:05.722400 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 605/1217] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:05:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:05:05.727611 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3328c4-bdd0-4c32-94b9-6278a962e86c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:05.844471 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3328c4-bdd0-4c32-94b9-6278a962e86c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:05.870427 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:05.890350 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e35b5a-6b0d-4346-a60b-1c057c61ea05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:05.892066 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e35b5a-6b0d-4346-a60b-1c057c61ea05 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:05.892666 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e35b5a-6b0d-4346-a60b-1c057c61ea05 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:05.893346 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e35b5a-6b0d-4346-a60b-1c057c61ea05 None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:05.894971 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e35b5a-6b0d-4346-a60b-1c057c61ea05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:05.895686 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e35b5a-6b0d-4346-a60b-1c057c61ea05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:05.897771 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e35b5a-6b0d-4346-a60b-1c057c61ea05 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:05.916372 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 613/1218] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jul 13 23:05:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:05:05.922577 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea45e3-fbef-411e-8d4c-a6de54e8b1d2 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:06.045530 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea45e3-fbef-411e-8d4c-a6de54e8b1d2 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:06.075386 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:06.083023 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dacd18e-03e8-47c5-8f35-a9882254146b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:06.083966 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dacd18e-03e8-47c5-8f35-a9882254146b None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:06.084339 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dacd18e-03e8-47c5-8f35-a9882254146b None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:06.084719 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dacd18e-03e8-47c5-8f35-a9882254146b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:06.085766 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dacd18e-03e8-47c5-8f35-a9882254146b None admin] RBAC: Adding query filter params (role.id=92417b14d4984ed9956dcd14f97564f3, scope.project.id=ab0a5581e4e444c293815e1304e20c05, user.id=64824ba84a024930bbc177f8a8d9a046) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:06.086303 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dacd18e-03e8-47c5-8f35-a9882254146b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:06.088082 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dacd18e-03e8-47c5-8f35-a9882254146b None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:06.100924 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 606/1219] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:05:05 2022] GET /identity/v3/role_assignments?user.id=64824ba84a024930bbc177f8a8d9a046&scope.project.id=ab0a5581e4e444c293815e1304e20c05&role.id=92417b14d4984ed9956dcd14f97564f3 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:05:08.172325 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3161d-526d-46eb-b730-fe68a604d647 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:08.172884 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3161d-526d-46eb-b730-fe68a604d647 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:08.172884 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3161d-526d-46eb-b730-fe68a604d647 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:08.174251 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 614/1220] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:05:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:05:08.179571 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939dd2e-d843-4c56-9ea3-5cd18bae8a54 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:08.180040 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939dd2e-d843-4c56-9ea3-5cd18bae8a54 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:08.180142 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939dd2e-d843-4c56-9ea3-5cd18bae8a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:08.197008 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-1939dd2e-d843-4c56-9ea3-5cd18bae8a54 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:05:08.198213 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-1939dd2e-d843-4c56-9ea3-5cd18bae8a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:05:08.281079 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 607/1221] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:05:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:05:08.302724 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3161d-526d-46eb-b730-fe68a604d647 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:08.414311 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3161d-526d-46eb-b730-fe68a604d647 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:08.452018 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2d055-7668-4a8e-b119-965bdc271209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:08.452648 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2d055-7668-4a8e-b119-965bdc271209 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:08.452922 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2d055-7668-4a8e-b119-965bdc271209 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:08.453227 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2d055-7668-4a8e-b119-965bdc271209 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:08.463298 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda2d055-7668-4a8e-b119-965bdc271209 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:05:08.471599 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-cda2d055-7668-4a8e-b119-965bdc271209 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:05:08.471599 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:05:08.471599 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:05:08.471599 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:05:08.471599 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:05:08.471599 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:05:08.471599 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:05:08.471599 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:05:08.471599 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:05:08.471599 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:05:08.471599 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:05:08.471599 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:05:08.471599 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:05:08.471599 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:05:08.471599 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:05:08.471599 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:05:08.472556 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:05:08.472556 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:05:08.472556 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:05:08.472556 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:05:08.472556 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:05:08.472556 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:05:08.472556 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:05:08.472556 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:05:08.472556 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:05:08.472556 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:05:08.472556 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:05:08.472556 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:05:08.472556 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:05:08.472556 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:05:08.472556 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:05:08.472556 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:05:08.473415 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:05:08.473415 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:05:08.473415 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:05:08.473415 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:05:08.473415 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:05:08.473415 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:05:08.473415 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:05:08.473415 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:05:08.473415 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:05:08.473415 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:05:08.473415 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:05:08.473415 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:05:08.473415 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 615/1222] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jul 13 23:05:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:05:08.478954 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939dd2e-d843-4c56-9ea3-5cd18bae8a54 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:08.583132 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939dd2e-d843-4c56-9ea3-5cd18bae8a54 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:08.610791 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:08.621602 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd992d3-f899-4c02-9a26-8662cd0377cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:08.622266 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd992d3-f899-4c02-9a26-8662cd0377cd None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:08.623044 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd992d3-f899-4c02-9a26-8662cd0377cd None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:08.623312 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd992d3-f899-4c02-9a26-8662cd0377cd None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:08.624090 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd992d3-f899-4c02-9a26-8662cd0377cd None admin] RBAC: Adding query filter params (name=service) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:08.624410 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd992d3-f899-4c02-9a26-8662cd0377cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:08.625793 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd992d3-f899-4c02-9a26-8662cd0377cd None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:08.634034 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 608/1223] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jul 13 23:05:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:05:08.640383 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2d055-7668-4a8e-b119-965bdc271209 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:08.685276 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:08.746370 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2d055-7668-4a8e-b119-965bdc271209 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:08.783550 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef130d3-3268-4204-968c-97472fdaee7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:08.784394 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef130d3-3268-4204-968c-97472fdaee7a None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:08.784725 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef130d3-3268-4204-968c-97472fdaee7a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:08.785011 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef130d3-3268-4204-968c-97472fdaee7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:08.796353 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef130d3-3268-4204-968c-97472fdaee7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:08.798881 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef130d3-3268-4204-968c-97472fdaee7a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:08.816285 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-6ef130d3-3268-4204-968c-97472fdaee7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:05:08.816285 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:05:08.816285 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:05:08.816285 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:05:08.816285 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:05:08.816285 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:05:08.816285 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:05:08.816285 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:05:08.816285 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:05:08.816285 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:05:08.816285 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:05:08.816285 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:05:08.816285 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:05:08.816285 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:05:08.816285 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:05:08.817376 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:05:08.817376 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:05:08.817376 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:05:08.817376 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:05:08.817376 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:05:08.817376 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:05:08.817376 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:05:08.817376 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:05:08.817376 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:05:08.817376 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:05:08.817376 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:05:08.817376 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:05:08.817376 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:05:08.817376 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:05:08.817376 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:05:08.817376 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:05:08.817376 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:05:08.818415 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:05:08.818415 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:05:08.818415 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:05:08.818415 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:05:08.818415 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 616/1224] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:05:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:05:08.823396 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd992d3-f899-4c02-9a26-8662cd0377cd None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:08.925290 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd992d3-f899-4c02-9a26-8662cd0377cd None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:08.960902 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964f18e-23f7-4418-970b-a8b80b785492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:08.961571 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964f18e-23f7-4418-970b-a8b80b785492 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:08.961850 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964f18e-23f7-4418-970b-a8b80b785492 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:08.962120 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964f18e-23f7-4418-970b-a8b80b785492 None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:08.962842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0964f18e-23f7-4418-970b-a8b80b785492 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:08.963146 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0964f18e-23f7-4418-970b-a8b80b785492 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:08.964561 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0964f18e-23f7-4418-970b-a8b80b785492 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:08.976792 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 609/1225] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:05:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:05:08.983572 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef130d3-3268-4204-968c-97472fdaee7a None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:09.094199 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:09.102193 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef130d3-3268-4204-968c-97472fdaee7a None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:09.140646 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed1f808-e404-43f9-939e-eb0d76235969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:09.141448 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed1f808-e404-43f9-939e-eb0d76235969 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:09.141917 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed1f808-e404-43f9-939e-eb0d76235969 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:09.142040 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed1f808-e404-43f9-939e-eb0d76235969 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:09.150009 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed1f808-e404-43f9-939e-eb0d76235969 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:09.151457 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed1f808-e404-43f9-939e-eb0d76235969 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:09.165951 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-0ed1f808-e404-43f9-939e-eb0d76235969 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 13 23:05:09.165951 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:05:09.165951 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:05:09.165951 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:05:09.165951 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:05:09.165951 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:05:09.165951 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:05:09.165951 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:05:09.165951 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:05:09.165951 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:05:09.165951 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:05:09.165951 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:05:09.165951 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 23:05:09.165951 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 23:05:09.165951 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 23:05:09.165951 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 23:05:09.171861 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:05:09.171861 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:05:09.171861 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 23:05:09.171861 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:05:09.171861 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 23:05:09.171861 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:05:09.171861 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:05:09.171861 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:05:09.171861 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:05:09.171861 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:05:09.171861 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:05:09.171861 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:05:09.171861 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:05:09.171861 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:05:09.171861 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:05:09.171861 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:05:09.171861 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:05:09.171861 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:05:09.173627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:05:09.173627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:05:09.173627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 23:05:09.173627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 23:05:09.173627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 23:05:09.173627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 23:05:09.173627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 23:05:09.173627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 23:05:09.173627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 13 23:05:09.173627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:05:09.173627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 617/1226] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:05:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:05:09.175413 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964f18e-23f7-4418-970b-a8b80b785492 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:09.224286 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:09.283135 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964f18e-23f7-4418-970b-a8b80b785492 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:09.323655 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2857ad-8dcc-466f-a011-52aa53db7416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:09.324333 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2857ad-8dcc-466f-a011-52aa53db7416 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:09.324616 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2857ad-8dcc-466f-a011-52aa53db7416 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:09.324891 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2857ad-8dcc-466f-a011-52aa53db7416 None admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:09.325774 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2857ad-8dcc-466f-a011-52aa53db7416 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:09.326081 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2857ad-8dcc-466f-a011-52aa53db7416 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:09.327588 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2857ad-8dcc-466f-a011-52aa53db7416 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:09.353230 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 610/1227] 10.222.0.60 () {56 vars in 1143 bytes} [Wed Jul 13 23:05:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:05:09.360039 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed1f808-e404-43f9-939e-eb0d76235969 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:09.492584 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed1f808-e404-43f9-939e-eb0d76235969 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:09.530936 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a39c94-e8cb-4507-83b0-500f3a2622e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:09.531754 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a39c94-e8cb-4507-83b0-500f3a2622e6 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:09.532037 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a39c94-e8cb-4507-83b0-500f3a2622e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:09.532313 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a39c94-e8cb-4507-83b0-500f3a2622e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:09.544223 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a39c94-e8cb-4507-83b0-500f3a2622e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:09.546376 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a39c94-e8cb-4507-83b0-500f3a2622e6 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:09.565187 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-a8a39c94-e8cb-4507-83b0-500f3a2622e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:05:09.565187 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:05:09.565187 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:05:09.565187 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:05:09.565187 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:05:09.565187 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:05:09.565187 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:05:09.565187 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:05:09.565187 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:05:09.565187 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:05:09.565187 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:05:09.565187 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:05:09.565187 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:05:09.565187 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:05:09.565187 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:05:09.566480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:05:09.566480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:05:09.566480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:05:09.566480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:05:09.566480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:05:09.566480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:05:09.566480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:05:09.566480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:05:09.566480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:05:09.566480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:05:09.566480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:05:09.566480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:05:09.566480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:05:09.566480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:05:09.566480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:05:09.566480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:05:09.566480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:05:09.567803 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:05:09.567803 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:05:09.567803 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:05:09.567803 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:05:09.567803 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 618/1228] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:05:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:05:09.572681 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2857ad-8dcc-466f-a011-52aa53db7416 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:09.692991 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2857ad-8dcc-466f-a011-52aa53db7416 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:09.695992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:09.736469 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7ff4f3-65b1-4a0c-b7ee-ab0ef0c3d640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:09.737144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ff4f3-65b1-4a0c-b7ee-ab0ef0c3d640 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:09.737445 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ff4f3-65b1-4a0c-b7ee-ab0ef0c3d640 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:09.737791 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7ff4f3-65b1-4a0c-b7ee-ab0ef0c3d640 None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:09.738619 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7ff4f3-65b1-4a0c-b7ee-ab0ef0c3d640 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:09.738992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7ff4f3-65b1-4a0c-b7ee-ab0ef0c3d640 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:09.740700 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7ff4f3-65b1-4a0c-b7ee-ab0ef0c3d640 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:09.755096 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 611/1229] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:05:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:05:09.760939 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a39c94-e8cb-4507-83b0-500f3a2622e6 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:09.772209 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:09.878138 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a39c94-e8cb-4507-83b0-500f3a2622e6 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:09.915014 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb4adb-dd1b-4f53-8acd-c6544e604896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:09.915848 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb4adb-dd1b-4f53-8acd-c6544e604896 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:09.916124 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb4adb-dd1b-4f53-8acd-c6544e604896 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:09.916399 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb4adb-dd1b-4f53-8acd-c6544e604896 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:09.923824 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedb4adb-dd1b-4f53-8acd-c6544e604896 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:09.925361 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedb4adb-dd1b-4f53-8acd-c6544e604896 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:09.940753 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-eedb4adb-dd1b-4f53-8acd-c6544e604896 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:05:09.940753 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:05:09.940753 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:05:09.940753 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:05:09.940753 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:05:09.940753 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:05:09.940753 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:05:09.940753 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:05:09.940753 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:05:09.940753 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:05:09.940753 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:05:09.940753 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:05:09.940753 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 23:05:09.940753 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 23:05:09.940753 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 23:05:09.942060 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 23:05:09.942060 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:05:09.942060 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:05:09.942060 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:05:09.942060 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:05:09.942060 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:05:09.942060 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:05:09.942060 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:05:09.942060 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:05:09.942060 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:05:09.942060 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:05:09.942060 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:05:09.942060 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:05:09.942060 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:05:09.942060 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:05:09.942060 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:05:09.942060 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:05:09.943151 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 23:05:09.943151 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 23:05:09.943151 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 23:05:09.943151 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 23:05:09.943151 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 23:05:09.943151 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 23:05:09.943151 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:05:09.943151 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:05:09.946814 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 619/1230] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:05:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 23:05:09.948080 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7ff4f3-65b1-4a0c-b7ee-ab0ef0c3d640 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:10.056762 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7ff4f3-65b1-4a0c-b7ee-ab0ef0c3d640 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:10.093976 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a6911-bd1c-4431-a3e1-8e1db6510259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:10.094925 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a6911-bd1c-4431-a3e1-8e1db6510259 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:10.095351 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a6911-bd1c-4431-a3e1-8e1db6510259 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:10.095824 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a6911-bd1c-4431-a3e1-8e1db6510259 None admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:10.096598 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a6911-bd1c-4431-a3e1-8e1db6510259 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:10.096898 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a6911-bd1c-4431-a3e1-8e1db6510259 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:10.098341 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a6911-bd1c-4431-a3e1-8e1db6510259 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:10.113840 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 612/1231] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jul 13 23:05:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:05:10.120738 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb4adb-dd1b-4f53-8acd-c6544e604896 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:10.196987 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:10.241402 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb4adb-dd1b-4f53-8acd-c6544e604896 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:10.280796 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f27c7-5fc1-468a-bd6b-28e7c6162e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:10.281700 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f27c7-5fc1-468a-bd6b-28e7c6162e2b None admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:10.281998 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f27c7-5fc1-468a-bd6b-28e7c6162e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:10.282263 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f27c7-5fc1-468a-bd6b-28e7c6162e2b None admin] PATH_INFO: `/v3/projects/ab0a5581e4e444c293815e1304e20c05/users/64824ba84a024930bbc177f8a8d9a046/roles/92417b14d4984ed9956dcd14f97564f3` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:10.300192 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f27c7-5fc1-468a-bd6b-28e7c6162e2b None admin] RBAC: Authorizing `identity:create_grant(project_id=ab0a5581e4e444c293815e1304e20c05, user_id=64824ba84a024930bbc177f8a8d9a046, role_id=92417b14d4984ed9956dcd14f97564f3)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:10.301818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f27c7-5fc1-468a-bd6b-28e7c6162e2b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:10.314363 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 620/1232] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jul 13 23:05:10 2022] PUT /identity/v3/projects/ab0a5581e4e444c293815e1304e20c05/users/64824ba84a024930bbc177f8a8d9a046/roles/92417b14d4984ed9956dcd14f97564f3 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:05:12.661402 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb1893-fac3-4d72-9553-e7db4d879a52 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:12.661965 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb1893-fac3-4d72-9553-e7db4d879a52 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:12.661965 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb1893-fac3-4d72-9553-e7db4d879a52 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:12.663367 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 613/1233] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:05:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:05:12.669383 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad856f55-a478-4217-8a74-7680fe010b79 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:12.669917 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad856f55-a478-4217-8a74-7680fe010b79 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:12.669917 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad856f55-a478-4217-8a74-7680fe010b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:12.686735 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-ad856f55-a478-4217-8a74-7680fe010b79 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:05:12.688173 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ad856f55-a478-4217-8a74-7680fe010b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:05:12.766927 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 621/1234] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:05:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 23:05:12.792088 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb1893-fac3-4d72-9553-e7db4d879a52 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:12.822402 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:12.916655 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb1893-fac3-4d72-9553-e7db4d879a52 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:12.953558 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4744680d-fa22-4245-bd42-8a4fbf55585c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:12.954451 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744680d-fa22-4245-bd42-8a4fbf55585c None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:12.954745 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744680d-fa22-4245-bd42-8a4fbf55585c None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:12.955028 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744680d-fa22-4245-bd42-8a4fbf55585c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:12.966368 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4744680d-fa22-4245-bd42-8a4fbf55585c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:05:12.975170 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-4744680d-fa22-4245-bd42-8a4fbf55585c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:05:12.975170 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:05:12.975170 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:05:12.975170 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:05:12.975170 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:05:12.975170 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:05:12.975170 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:05:12.975170 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:05:12.975170 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:05:12.975170 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:05:12.975170 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:05:12.975170 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:05:12.975170 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:05:12.975170 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:05:12.975170 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:05:12.975170 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:05:12.976426 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:05:12.976426 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:05:12.976426 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:05:12.976426 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:05:12.976426 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:05:12.976426 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:05:12.976426 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:05:12.976426 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:05:12.976426 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:05:12.976426 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:05:12.976426 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:05:12.976426 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:05:12.976426 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:05:12.976426 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:05:12.976426 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:05:12.976426 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:05:12.977584 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:05:12.977584 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:05:12.977584 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:05:12.977584 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:05:12.977584 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:05:12.977584 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:05:12.977584 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:05:12.977584 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:05:12.977584 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:05:12.977584 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:05:12.977584 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 23:05:12.977584 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:05:12.979856 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 614/1235] 10.222.0.60 () {56 vars in 1102 bytes} [Wed Jul 13 23:05:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:05:12.986932 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad856f55-a478-4217-8a74-7680fe010b79 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:13.047311 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:13.111422 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad856f55-a478-4217-8a74-7680fe010b79 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:13.151662 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f9200-ad89-4da5-a2c2-727e308f00c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:13.152385 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f9200-ad89-4da5-a2c2-727e308f00c2 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:13.152721 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f9200-ad89-4da5-a2c2-727e308f00c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:13.153020 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f9200-ad89-4da5-a2c2-727e308f00c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:13.154436 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f9200-ad89-4da5-a2c2-727e308f00c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:13.154799 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f9200-ad89-4da5-a2c2-727e308f00c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:13.156321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f9200-ad89-4da5-a2c2-727e308f00c2 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:13.165981 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 622/1236] 10.222.0.60 () {56 vars in 1103 bytes} [Wed Jul 13 23:05:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:05:13.173221 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4744680d-fa22-4245-bd42-8a4fbf55585c None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:13.276217 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:13.293953 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4744680d-fa22-4245-bd42-8a4fbf55585c None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:13.331276 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713bd6ae-1d62-430d-98fa-c3c2df7c0bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:13.332162 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713bd6ae-1d62-430d-98fa-c3c2df7c0bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:13.332429 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713bd6ae-1d62-430d-98fa-c3c2df7c0bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:13.332711 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713bd6ae-1d62-430d-98fa-c3c2df7c0bc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:13.345377 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713bd6ae-1d62-430d-98fa-c3c2df7c0bc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:13.347037 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713bd6ae-1d62-430d-98fa-c3c2df7c0bc3 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:13.372650 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-713bd6ae-1d62-430d-98fa-c3c2df7c0bc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:05:13.372650 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:05:13.372650 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:05:13.372650 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:05:13.372650 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:05:13.372650 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:05:13.372650 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:05:13.372650 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:05:13.372650 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:05:13.372650 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:05:13.372650 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:05:13.372650 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:05:13.372650 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:05:13.372650 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:05:13.372650 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:05:13.373713 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:05:13.373713 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:05:13.373713 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:05:13.373713 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:05:13.373713 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:05:13.373713 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:05:13.373713 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:05:13.373713 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:05:13.373713 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:05:13.373713 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:05:13.373713 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:05:13.373713 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:05:13.373713 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:05:13.373713 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:05:13.373713 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:05:13.373713 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:05:13.373713 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:05:13.374918 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:05:13.374918 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:05:13.374918 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:05:13.374918 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:05:13.374918 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 615/1237] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:05:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:05:13.379988 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f9200-ad89-4da5-a2c2-727e308f00c2 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:13.493408 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f9200-ad89-4da5-a2c2-727e308f00c2 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:13.504254 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:13.533131 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6e263-b865-41c0-9471-cfa20ceb0ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:13.533728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6e263-b865-41c0-9471-cfa20ceb0ba6 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:13.533995 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6e263-b865-41c0-9471-cfa20ceb0ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:13.534271 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6e263-b865-41c0-9471-cfa20ceb0ba6 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:13.535012 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a6e263-b865-41c0-9471-cfa20ceb0ba6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:13.535320 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a6e263-b865-41c0-9471-cfa20ceb0ba6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:13.536772 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a6e263-b865-41c0-9471-cfa20ceb0ba6 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:13.549630 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 623/1238] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:05:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:05:13.555907 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713bd6ae-1d62-430d-98fa-c3c2df7c0bc3 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:13.667703 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713bd6ae-1d62-430d-98fa-c3c2df7c0bc3 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:13.703379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3207da64-3eac-4773-8787-c90935dc0f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:13.704441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3207da64-3eac-4773-8787-c90935dc0f4b None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:13.704839 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3207da64-3eac-4773-8787-c90935dc0f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:13.705184 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3207da64-3eac-4773-8787-c90935dc0f4b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:13.708333 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:13.713915 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3207da64-3eac-4773-8787-c90935dc0f4b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:13.715726 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3207da64-3eac-4773-8787-c90935dc0f4b None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:13.732758 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-3207da64-3eac-4773-8787-c90935dc0f4b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 13 23:05:13.732758 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:05:13.732758 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:05:13.732758 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:05:13.732758 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:05:13.732758 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:05:13.732758 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:05:13.732758 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:05:13.732758 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:05:13.732758 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:05:13.732758 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:05:13.732758 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:05:13.732758 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 23:05:13.732758 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 23:05:13.732758 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 23:05:13.732758 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 23:05:13.734172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:05:13.734172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:05:13.734172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 23:05:13.734172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:05:13.734172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 23:05:13.734172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:05:13.734172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:05:13.734172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:05:13.734172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:05:13.734172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:05:13.734172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:05:13.734172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:05:13.734172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:05:13.734172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:05:13.734172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:05:13.734172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:05:13.734172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:05:13.734172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:05:13.735424 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:05:13.735424 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:05:13.735424 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 23:05:13.735424 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 23:05:13.735424 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 23:05:13.735424 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 23:05:13.735424 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 23:05:13.735424 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 23:05:13.735424 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 13 23:05:13.735424 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:05:13.735424 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 616/1239] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:05:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:05:13.742828 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6e263-b865-41c0-9471-cfa20ceb0ba6 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:13.853005 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6e263-b865-41c0-9471-cfa20ceb0ba6 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:13.889811 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf91091d-c9b2-439a-9351-2b5f07608ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:13.890505 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf91091d-c9b2-439a-9351-2b5f07608ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:13.890791 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf91091d-c9b2-439a-9351-2b5f07608ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:13.891068 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf91091d-c9b2-439a-9351-2b5f07608ec6 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:13.892051 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf91091d-c9b2-439a-9351-2b5f07608ec6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:13.892364 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf91091d-c9b2-439a-9351-2b5f07608ec6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:13.893988 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf91091d-c9b2-439a-9351-2b5f07608ec6 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:13.913375 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 624/1240] 10.222.0.60 () {56 vars in 1143 bytes} [Wed Jul 13 23:05:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:05:13.919713 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3207da64-3eac-4773-8787-c90935dc0f4b None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:14.029967 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3207da64-3eac-4773-8787-c90935dc0f4b None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:14.067672 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6572e-a4c7-4d53-9465-193dac6d9c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:14.068331 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6572e-a4c7-4d53-9465-193dac6d9c65 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:14.068616 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6572e-a4c7-4d53-9465-193dac6d9c65 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:14.068896 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6572e-a4c7-4d53-9465-193dac6d9c65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:14.166378 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:14.176265 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e6572e-a4c7-4d53-9465-193dac6d9c65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:14.177931 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e6572e-a4c7-4d53-9465-193dac6d9c65 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:14.296068 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 617/1241] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:05:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 23:05:14.302098 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf91091d-c9b2-439a-9351-2b5f07608ec6 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:14.333093 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:14.421119 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf91091d-c9b2-439a-9351-2b5f07608ec6 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:14.456657 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482df4c8-b54c-432c-98df-cb2e6a1ee2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:14.457449 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482df4c8-b54c-432c-98df-cb2e6a1ee2dd None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:14.457732 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482df4c8-b54c-432c-98df-cb2e6a1ee2dd None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:14.458007 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482df4c8-b54c-432c-98df-cb2e6a1ee2dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:14.468376 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482df4c8-b54c-432c-98df-cb2e6a1ee2dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:14.470087 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482df4c8-b54c-432c-98df-cb2e6a1ee2dd None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:14.489776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-482df4c8-b54c-432c-98df-cb2e6a1ee2dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:05:14.489776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:05:14.489776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:05:14.489776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:05:14.489776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:05:14.489776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:05:14.489776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:05:14.489776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:05:14.489776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:05:14.489776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:05:14.489776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:05:14.489776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:05:14.489776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:05:14.489776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:05:14.489776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:05:14.490807 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:05:14.490807 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:05:14.490807 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:05:14.490807 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:05:14.490807 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:05:14.490807 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:05:14.490807 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:05:14.490807 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:05:14.490807 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:05:14.490807 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:05:14.490807 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:05:14.490807 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:05:14.490807 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:05:14.490807 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:05:14.490807 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:05:14.490807 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:05:14.490807 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:05:14.492236 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:05:14.492236 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:05:14.492236 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:05:14.492236 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:05:14.492236 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 625/1242] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:05:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:05:14.496897 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6572e-a4c7-4d53-9465-193dac6d9c65 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:14.598823 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:14.605858 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6572e-a4c7-4d53-9465-193dac6d9c65 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:14.647397 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2018412e-09d1-458a-a85f-efd83ab9f681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:14.648130 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2018412e-09d1-458a-a85f-efd83ab9f681 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:14.648412 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2018412e-09d1-458a-a85f-efd83ab9f681 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:14.648687 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2018412e-09d1-458a-a85f-efd83ab9f681 None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:14.649413 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2018412e-09d1-458a-a85f-efd83ab9f681 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:14.649721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2018412e-09d1-458a-a85f-efd83ab9f681 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:14.651098 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2018412e-09d1-458a-a85f-efd83ab9f681 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:14.665031 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 618/1243] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:05:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:05:14.671500 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482df4c8-b54c-432c-98df-cb2e6a1ee2dd None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:14.756339 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:14.781162 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482df4c8-b54c-432c-98df-cb2e6a1ee2dd None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:14.820475 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c11299f-d63a-4324-be09-c50352208390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:14.821304 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11299f-d63a-4324-be09-c50352208390 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:14.821582 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11299f-d63a-4324-be09-c50352208390 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:14.821853 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11299f-d63a-4324-be09-c50352208390 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:14.828963 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c11299f-d63a-4324-be09-c50352208390 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:14.830825 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c11299f-d63a-4324-be09-c50352208390 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:14.847665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-9c11299f-d63a-4324-be09-c50352208390 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:05:14.847665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:05:14.847665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:05:14.847665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:05:14.847665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:05:14.847665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:05:14.847665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:05:14.847665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:05:14.847665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:05:14.847665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:05:14.847665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:05:14.847665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:05:14.847665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 23:05:14.847665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 23:05:14.847665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 23:05:14.848835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 23:05:14.848835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:05:14.848835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:05:14.848835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:05:14.848835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:05:14.848835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:05:14.848835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:05:14.848835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:05:14.848835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:05:14.848835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:05:14.848835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:05:14.848835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:05:14.848835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:05:14.848835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:05:14.848835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:05:14.848835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:05:14.848835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:05:14.849938 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 23:05:14.849938 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 23:05:14.849938 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 23:05:14.849938 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 23:05:14.849938 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 23:05:14.849938 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 23:05:14.849938 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:05:14.849938 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:05:14.854770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 626/1244] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:05:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:05:14.855809 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2018412e-09d1-458a-a85f-efd83ab9f681 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:14.967025 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2018412e-09d1-458a-a85f-efd83ab9f681 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:15.002515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13db461-9a66-4a8d-abee-132fda83f9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:15.003177 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13db461-9a66-4a8d-abee-132fda83f9ad None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:15.003457 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13db461-9a66-4a8d-abee-132fda83f9ad None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:15.003759 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13db461-9a66-4a8d-abee-132fda83f9ad None admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:15.004581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13db461-9a66-4a8d-abee-132fda83f9ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:15.004887 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13db461-9a66-4a8d-abee-132fda83f9ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:15.006450 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13db461-9a66-4a8d-abee-132fda83f9ad None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:15.020460 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 619/1245] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jul 13 23:05:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:05:15.026756 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c11299f-d63a-4324-be09-c50352208390 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:15.145951 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c11299f-d63a-4324-be09-c50352208390 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:15.181542 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f5d88-43c6-48e2-acd1-d7563c8d4454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:15.182208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f5d88-43c6-48e2-acd1-d7563c8d4454 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:15.182498 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f5d88-43c6-48e2-acd1-d7563c8d4454 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:15.182758 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f5d88-43c6-48e2-acd1-d7563c8d4454 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:15.183629 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f5d88-43c6-48e2-acd1-d7563c8d4454 None admin] RBAC: Adding query filter params (role.id=92417b14d4984ed9956dcd14f97564f3, scope.project.id=ab0a5581e4e444c293815e1304e20c05, user.id=64824ba84a024930bbc177f8a8d9a046) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:15.183960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f5d88-43c6-48e2-acd1-d7563c8d4454 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:15.185374 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f5d88-43c6-48e2-acd1-d7563c8d4454 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:15.192360 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 627/1246] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:05:15 2022] GET /identity/v3/role_assignments?user.id=64824ba84a024930bbc177f8a8d9a046&scope.project.id=ab0a5581e4e444c293815e1304e20c05&role.id=92417b14d4984ed9956dcd14f97564f3 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:05:17.414883 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf01c6-9cec-4632-aa32-6bf2c31a2632 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:17.415430 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf01c6-9cec-4632-aa32-6bf2c31a2632 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:17.415430 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf01c6-9cec-4632-aa32-6bf2c31a2632 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:17.417096 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 620/1247] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:05:17.423330 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee4485-11ec-48e1-b6d6-143a00b3be11 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:17.424068 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee4485-11ec-48e1-b6d6-143a00b3be11 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:17.424068 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee4485-11ec-48e1-b6d6-143a00b3be11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:17.429578 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:17.444044 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-45ee4485-11ec-48e1-b6d6-143a00b3be11 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:05:17.445235 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-45ee4485-11ec-48e1-b6d6-143a00b3be11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:05:17.522834 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 628/1248] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:05:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:05:17.546895 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf01c6-9cec-4632-aa32-6bf2c31a2632 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:17.567720 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:17.659375 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf01c6-9cec-4632-aa32-6bf2c31a2632 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:17.699686 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38967d7-ea20-410c-9b78-b21363c6e62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:17.700525 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38967d7-ea20-410c-9b78-b21363c6e62c None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:17.700793 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38967d7-ea20-410c-9b78-b21363c6e62c None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:17.701067 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38967d7-ea20-410c-9b78-b21363c6e62c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:17.712695 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38967d7-ea20-410c-9b78-b21363c6e62c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:05:17.721233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-e38967d7-ea20-410c-9b78-b21363c6e62c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:05:17.721233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:05:17.721233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:05:17.721233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:05:17.721233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:05:17.721233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:05:17.721233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:05:17.721233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:05:17.721233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:05:17.721233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:05:17.721233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:05:17.721233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:05:17.721233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:05:17.721233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:05:17.721233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:05:17.721233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:05:17.722544 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:05:17.722544 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:05:17.722544 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:05:17.722544 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:05:17.722544 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:05:17.722544 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:05:17.722544 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:05:17.722544 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:05:17.722544 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:05:17.722544 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:05:17.722544 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:05:17.722544 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:05:17.722544 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:05:17.722544 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:05:17.722544 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:05:17.722544 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:05:17.723462 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:05:17.723462 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:05:17.723462 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:05:17.723462 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:05:17.723462 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:05:17.723462 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:05:17.723462 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:05:17.723462 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:05:17.723462 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:05:17.723462 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:05:17.723462 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:05:17.723462 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:05:17.723462 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 621/1249] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jul 13 23:05:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:05:17.730781 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ee4485-11ec-48e1-b6d6-143a00b3be11 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:17.843606 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ee4485-11ec-48e1-b6d6-143a00b3be11 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:17.875736 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:17.881372 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9a385-d99f-40f4-898d-ca8425fe111f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:17.882216 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9a385-d99f-40f4-898d-ca8425fe111f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:17.882713 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9a385-d99f-40f4-898d-ca8425fe111f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:17.882968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9a385-d99f-40f4-898d-ca8425fe111f None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:17.883752 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f9a385-d99f-40f4-898d-ca8425fe111f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:17.884048 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f9a385-d99f-40f4-898d-ca8425fe111f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:17.885449 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f9a385-d99f-40f4-898d-ca8425fe111f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:17.894389 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 629/1250] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:05:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:05:17.900600 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38967d7-ea20-410c-9b78-b21363c6e62c None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:17.988633 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:18.020412 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38967d7-ea20-410c-9b78-b21363c6e62c None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:18.058148 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3427df4-d605-416d-9460-c52658b6f055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:18.059126 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3427df4-d605-416d-9460-c52658b6f055 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:18.059639 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3427df4-d605-416d-9460-c52658b6f055 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:18.060016 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3427df4-d605-416d-9460-c52658b6f055 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:18.071652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3427df4-d605-416d-9460-c52658b6f055 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:18.073125 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3427df4-d605-416d-9460-c52658b6f055 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:18.096998 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-f3427df4-d605-416d-9460-c52658b6f055 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:05:18.096998 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:05:18.096998 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:05:18.096998 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:05:18.096998 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:05:18.096998 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:05:18.096998 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:05:18.096998 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:05:18.096998 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:05:18.096998 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:05:18.096998 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:05:18.096998 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:05:18.096998 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:05:18.096998 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:05:18.096998 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:05:18.098712 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:05:18.098712 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:05:18.098712 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:05:18.098712 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:05:18.098712 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:05:18.098712 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:05:18.098712 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:05:18.098712 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:05:18.098712 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:05:18.098712 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:05:18.098712 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:05:18.098712 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:05:18.098712 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:05:18.098712 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:05:18.098712 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:05:18.098712 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:05:18.098712 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:05:18.099655 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:05:18.099655 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:05:18.099655 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:05:18.099655 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:05:18.099655 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 622/1251] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:05:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:05:18.104876 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9a385-d99f-40f4-898d-ca8425fe111f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:18.225395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9a385-d99f-40f4-898d-ca8425fe111f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:18.265825 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfbebb9-9a5e-44e1-bb1f-05ab588eddce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:18.266414 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfbebb9-9a5e-44e1-bb1f-05ab588eddce None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:18.266697 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfbebb9-9a5e-44e1-bb1f-05ab588eddce None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:18.266972 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfbebb9-9a5e-44e1-bb1f-05ab588eddce None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:18.267742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfbebb9-9a5e-44e1-bb1f-05ab588eddce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:18.268096 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfbebb9-9a5e-44e1-bb1f-05ab588eddce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:18.269640 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfbebb9-9a5e-44e1-bb1f-05ab588eddce None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:18.284339 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 630/1252] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:05:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:05:18.290182 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3427df4-d605-416d-9460-c52658b6f055 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:18.403035 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3427df4-d605-416d-9460-c52658b6f055 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:18.430461 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:18.441157 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a324f8a-645d-4fb5-9086-a94e295028a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:18.441984 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a324f8a-645d-4fb5-9086-a94e295028a9 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:18.442249 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a324f8a-645d-4fb5-9086-a94e295028a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:18.442542 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a324f8a-645d-4fb5-9086-a94e295028a9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:18.451027 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a324f8a-645d-4fb5-9086-a94e295028a9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:18.452543 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a324f8a-645d-4fb5-9086-a94e295028a9 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:18.467948 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-8a324f8a-645d-4fb5-9086-a94e295028a9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 13 23:05:18.467948 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:05:18.467948 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:05:18.467948 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:05:18.467948 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:05:18.467948 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:05:18.467948 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:05:18.467948 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:05:18.467948 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:05:18.467948 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:05:18.467948 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:05:18.467948 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:05:18.467948 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 23:05:18.467948 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 23:05:18.467948 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 23:05:18.467948 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 23:05:18.469051 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:05:18.469051 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:05:18.469051 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 23:05:18.469051 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:05:18.469051 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 23:05:18.469051 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:05:18.469051 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:05:18.469051 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:05:18.469051 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:05:18.469051 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:05:18.469051 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:05:18.469051 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:05:18.469051 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:05:18.469051 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:05:18.469051 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:05:18.469051 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:05:18.469051 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:05:18.469051 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:05:18.470631 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:05:18.470631 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:05:18.470631 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 23:05:18.470631 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 23:05:18.470631 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 23:05:18.470631 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 23:05:18.470631 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 23:05:18.470631 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 23:05:18.470631 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 13 23:05:18.470631 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:05:18.470631 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 623/1253] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:05:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:05:18.475374 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfbebb9-9a5e-44e1-bb1f-05ab588eddce None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:18.538769 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:18.593180 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfbebb9-9a5e-44e1-bb1f-05ab588eddce None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:18.631735 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f4cc33-6a5c-492b-b30c-771e5321350b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:18.632402 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4cc33-6a5c-492b-b30c-771e5321350b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:18.632680 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4cc33-6a5c-492b-b30c-771e5321350b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:18.633557 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4cc33-6a5c-492b-b30c-771e5321350b None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:18.634429 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f4cc33-6a5c-492b-b30c-771e5321350b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:18.634735 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f4cc33-6a5c-492b-b30c-771e5321350b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:18.636495 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f4cc33-6a5c-492b-b30c-771e5321350b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:18.656837 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 631/1254] 10.222.0.60 () {56 vars in 1143 bytes} [Wed Jul 13 23:05:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:05:18.663092 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a324f8a-645d-4fb5-9086-a94e295028a9 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:18.774783 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a324f8a-645d-4fb5-9086-a94e295028a9 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:18.811126 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0988a958-5003-44c1-90e7-c0e90a2f44a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:18.811732 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988a958-5003-44c1-90e7-c0e90a2f44a1 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:18.812018 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988a958-5003-44c1-90e7-c0e90a2f44a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:18.812295 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988a958-5003-44c1-90e7-c0e90a2f44a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:18.874368 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:18.886806 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0988a958-5003-44c1-90e7-c0e90a2f44a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:18.888365 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0988a958-5003-44c1-90e7-c0e90a2f44a1 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:18.998094 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 624/1255] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:05:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 13 23:05:19.004159 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f4cc33-6a5c-492b-b30c-771e5321350b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:19.112914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f4cc33-6a5c-492b-b30c-771e5321350b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:19.129848 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:19.155694 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c375da69-9d85-4c61-a998-1f77e014cb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:19.156560 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375da69-9d85-4c61-a998-1f77e014cb5b None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:19.156848 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375da69-9d85-4c61-a998-1f77e014cb5b None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:19.157155 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375da69-9d85-4c61-a998-1f77e014cb5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:19.169984 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c375da69-9d85-4c61-a998-1f77e014cb5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:19.171448 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c375da69-9d85-4c61-a998-1f77e014cb5b None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:19.191365 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-c375da69-9d85-4c61-a998-1f77e014cb5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:05:19.191365 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:05:19.191365 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:05:19.191365 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:05:19.191365 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:05:19.191365 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:05:19.191365 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:05:19.191365 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:05:19.191365 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:05:19.191365 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:05:19.191365 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:05:19.191365 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:05:19.191365 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:05:19.191365 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:05:19.191365 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:05:19.192504 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:05:19.192504 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:05:19.192504 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:05:19.192504 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:05:19.192504 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:05:19.192504 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:05:19.192504 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:05:19.192504 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:05:19.192504 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:05:19.192504 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:05:19.192504 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:05:19.192504 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:05:19.192504 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:05:19.192504 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:05:19.192504 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:05:19.192504 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:05:19.192504 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:05:19.193478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:05:19.193478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:05:19.193478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:05:19.193478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:05:19.193478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 632/1256] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:05:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:05:19.198830 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0988a958-5003-44c1-90e7-c0e90a2f44a1 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:19.310087 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:19.323823 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0988a958-5003-44c1-90e7-c0e90a2f44a1 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:19.365105 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aef45a-a4bc-4666-9c7b-d83cdfd8a7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:19.365801 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aef45a-a4bc-4666-9c7b-d83cdfd8a7fe None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:19.366090 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aef45a-a4bc-4666-9c7b-d83cdfd8a7fe None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:19.366370 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aef45a-a4bc-4666-9c7b-d83cdfd8a7fe None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:19.367145 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aef45a-a4bc-4666-9c7b-d83cdfd8a7fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:19.367493 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aef45a-a4bc-4666-9c7b-d83cdfd8a7fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:19.368874 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aef45a-a4bc-4666-9c7b-d83cdfd8a7fe None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:19.381656 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 625/1257] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:05:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:05:19.387438 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c375da69-9d85-4c61-a998-1f77e014cb5b None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:19.502086 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c375da69-9d85-4c61-a998-1f77e014cb5b None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:19.547496 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a522bb-21ef-4c28-86f6-088bb140b1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:19.548603 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a522bb-21ef-4c28-86f6-088bb140b1bc None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:19.548901 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a522bb-21ef-4c28-86f6-088bb140b1bc None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:19.549162 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a522bb-21ef-4c28-86f6-088bb140b1bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:19.556386 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a522bb-21ef-4c28-86f6-088bb140b1bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:19.557927 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a522bb-21ef-4c28-86f6-088bb140b1bc None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:19.573071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-a9a522bb-21ef-4c28-86f6-088bb140b1bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:05:19.573071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:05:19.573071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:05:19.573071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:05:19.573071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:05:19.573071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:05:19.573071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:05:19.573071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:05:19.573071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:05:19.573071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:05:19.573071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:05:19.573071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:05:19.573071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 23:05:19.573071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 23:05:19.573071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 23:05:19.574097 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 23:05:19.574097 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:05:19.574097 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:05:19.574097 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:05:19.574097 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:05:19.574097 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:05:19.574097 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:05:19.574097 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:05:19.574097 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:05:19.574097 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:05:19.574097 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:05:19.574097 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:05:19.574097 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:05:19.574097 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:05:19.574097 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:05:19.574097 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:05:19.574097 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:05:19.575503 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 23:05:19.575503 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 23:05:19.575503 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 23:05:19.575503 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 23:05:19.575503 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 23:05:19.575503 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 23:05:19.575503 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:05:19.575503 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:05:19.575503 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 633/1258] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:05:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:05:19.580704 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aef45a-a4bc-4666-9c7b-d83cdfd8a7fe None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:19.690158 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aef45a-a4bc-4666-9c7b-d83cdfd8a7fe None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:19.735780 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29d380-b23e-4492-a30d-e7992f17ee1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:19.736454 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29d380-b23e-4492-a30d-e7992f17ee1a None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:19.736736 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29d380-b23e-4492-a30d-e7992f17ee1a None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:19.737010 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29d380-b23e-4492-a30d-e7992f17ee1a None admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:19.737812 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae29d380-b23e-4492-a30d-e7992f17ee1a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:19.738127 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae29d380-b23e-4492-a30d-e7992f17ee1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:19.739830 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae29d380-b23e-4492-a30d-e7992f17ee1a None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:19.756297 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 626/1259] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jul 13 23:05:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:05:19.761424 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a522bb-21ef-4c28-86f6-088bb140b1bc None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:19.792080 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:19.873102 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a522bb-21ef-4c28-86f6-088bb140b1bc None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:19.910548 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d744b0-4a22-404b-b182-67be752fb7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:19.911205 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d744b0-4a22-404b-b182-67be752fb7a6 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:19.911488 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d744b0-4a22-404b-b182-67be752fb7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:19.911796 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d744b0-4a22-404b-b182-67be752fb7a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:19.912626 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d744b0-4a22-404b-b182-67be752fb7a6 None admin] RBAC: Adding query filter params (role.id=8cfd0b6e71e14e0983c3eb4d7478e027, scope.project.id=ab0a5581e4e444c293815e1304e20c05, user.id=64824ba84a024930bbc177f8a8d9a046) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:19.912938 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d744b0-4a22-404b-b182-67be752fb7a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:19.914384 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d744b0-4a22-404b-b182-67be752fb7a6 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:19.921196 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 634/1260] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:05:19 2022] GET /identity/v3/role_assignments?user.id=64824ba84a024930bbc177f8a8d9a046&scope.project.id=ab0a5581e4e444c293815e1304e20c05&role.id=8cfd0b6e71e14e0983c3eb4d7478e027 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:05:21.989486 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca60529e-a3d0-488f-ba0a-1b536344945a None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:21.990053 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca60529e-a3d0-488f-ba0a-1b536344945a None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:21.990053 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca60529e-a3d0-488f-ba0a-1b536344945a None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:21.991406 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 627/1261] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:05:21.996685 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0546812-92d2-4cea-a711-2dd91364f707 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:21.997106 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0546812-92d2-4cea-a711-2dd91364f707 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:21.997233 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0546812-92d2-4cea-a711-2dd91364f707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:22.013912 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-a0546812-92d2-4cea-a711-2dd91364f707 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:05:22.015126 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a0546812-92d2-4cea-a711-2dd91364f707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:05:22.079075 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 635/1262] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:05:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 23:05:22.107128 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca60529e-a3d0-488f-ba0a-1b536344945a None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:22.118428 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:22.209459 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca60529e-a3d0-488f-ba0a-1b536344945a None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:22.242852 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb859b-1474-4db3-aa81-5e2776dfb911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:22.243568 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb859b-1474-4db3-aa81-5e2776dfb911 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:22.243858 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb859b-1474-4db3-aa81-5e2776dfb911 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:22.244137 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb859b-1474-4db3-aa81-5e2776dfb911 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:22.254881 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb859b-1474-4db3-aa81-5e2776dfb911 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:05:22.263064 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-77bb859b-1474-4db3-aa81-5e2776dfb911 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:05:22.263064 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:05:22.263064 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:05:22.263064 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:05:22.263064 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:05:22.263064 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:05:22.263064 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:05:22.263064 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:05:22.263064 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:05:22.263064 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:05:22.263064 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:05:22.263064 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:05:22.263064 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:05:22.263064 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:05:22.263064 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:05:22.263064 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:05:22.264217 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:05:22.264217 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:05:22.264217 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:05:22.264217 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:05:22.264217 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:05:22.264217 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:05:22.264217 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:05:22.264217 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:05:22.264217 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:05:22.264217 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:05:22.264217 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:05:22.264217 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:05:22.264217 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:05:22.264217 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:05:22.264217 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:05:22.264217 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:05:22.265130 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:05:22.265130 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:05:22.265130 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:05:22.265130 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:05:22.265130 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:05:22.265130 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:05:22.265130 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:05:22.265130 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:05:22.265130 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:05:22.265130 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:05:22.265130 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:05:22.265130 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:05:22.265130 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 628/1263] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jul 13 23:05:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:05:22.270485 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0546812-92d2-4cea-a711-2dd91364f707 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:22.288483 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:22.369018 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0546812-92d2-4cea-a711-2dd91364f707 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:22.402345 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a08fe-91b3-4964-8d58-96d39899cb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:22.402999 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a08fe-91b3-4964-8d58-96d39899cb8c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:22.403277 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a08fe-91b3-4964-8d58-96d39899cb8c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:22.403573 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a08fe-91b3-4964-8d58-96d39899cb8c None admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:22.404310 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a08fe-91b3-4964-8d58-96d39899cb8c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:22.404610 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a08fe-91b3-4964-8d58-96d39899cb8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:22.405966 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a08fe-91b3-4964-8d58-96d39899cb8c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:22.413852 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 636/1264] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:05:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:05:22.419898 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb859b-1474-4db3-aa81-5e2776dfb911 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:22.489793 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:22.526314 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb859b-1474-4db3-aa81-5e2776dfb911 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:22.565199 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8f1e5-d458-41f5-93d5-0211fde55bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:22.565999 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8f1e5-d458-41f5-93d5-0211fde55bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:22.566277 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8f1e5-d458-41f5-93d5-0211fde55bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:22.566548 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8f1e5-d458-41f5-93d5-0211fde55bc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:22.578234 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b8f1e5-d458-41f5-93d5-0211fde55bc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:22.579753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b8f1e5-d458-41f5-93d5-0211fde55bc9 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:22.597508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-87b8f1e5-d458-41f5-93d5-0211fde55bc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:05:22.597508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:05:22.597508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:05:22.597508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:05:22.597508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:05:22.597508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:05:22.597508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:05:22.597508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:05:22.597508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:05:22.597508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:05:22.597508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:05:22.597508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:05:22.597508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:05:22.597508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:05:22.597508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:05:22.598652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:05:22.598652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:05:22.598652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:05:22.598652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:05:22.598652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:05:22.598652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:05:22.598652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:05:22.598652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:05:22.598652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:05:22.598652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:05:22.598652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:05:22.598652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:05:22.598652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:05:22.598652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:05:22.598652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:05:22.598652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:05:22.598652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:05:22.599718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:05:22.599718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:05:22.599718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:05:22.599718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:05:22.599718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 629/1265] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:05:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:05:22.604886 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a08fe-91b3-4964-8d58-96d39899cb8c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:22.692187 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:22.712459 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a08fe-91b3-4964-8d58-96d39899cb8c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:22.745042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc7043-9678-476a-913d-9562f314d06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:22.745695 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc7043-9678-476a-913d-9562f314d06f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:22.745980 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc7043-9678-476a-913d-9562f314d06f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:22.746254 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc7043-9678-476a-913d-9562f314d06f None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:22.747557 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fc7043-9678-476a-913d-9562f314d06f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:22.747865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fc7043-9678-476a-913d-9562f314d06f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:22.749230 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fc7043-9678-476a-913d-9562f314d06f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:22.760701 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 637/1266] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:05:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:05:22.766496 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8f1e5-d458-41f5-93d5-0211fde55bc9 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:22.862964 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8f1e5-d458-41f5-93d5-0211fde55bc9 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:22.882436 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:22.896191 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e156f1-08aa-4648-a415-cb45819ea3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:22.896986 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e156f1-08aa-4648-a415-cb45819ea3e2 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:22.897268 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e156f1-08aa-4648-a415-cb45819ea3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:22.897540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e156f1-08aa-4648-a415-cb45819ea3e2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:22.905393 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e156f1-08aa-4648-a415-cb45819ea3e2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:22.906849 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e156f1-08aa-4648-a415-cb45819ea3e2 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:22.921314 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-83e156f1-08aa-4648-a415-cb45819ea3e2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 13 23:05:22.921314 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:05:22.921314 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:05:22.921314 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:05:22.921314 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:05:22.921314 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:05:22.921314 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:05:22.921314 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:05:22.921314 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:05:22.921314 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:05:22.921314 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:05:22.921314 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:05:22.921314 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 23:05:22.921314 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 23:05:22.921314 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 23:05:22.921314 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 23:05:22.922379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:05:22.922379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:05:22.922379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 23:05:22.922379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:05:22.922379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 23:05:22.922379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:05:22.922379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:05:22.922379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:05:22.922379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:05:22.922379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:05:22.922379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:05:22.922379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:05:22.922379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:05:22.922379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:05:22.922379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:05:22.922379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:05:22.922379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:05:22.922379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:05:22.923602 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:05:22.923602 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:05:22.923602 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 23:05:22.923602 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 23:05:22.923602 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 23:05:22.923602 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 23:05:22.923602 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 23:05:22.923602 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 23:05:22.923602 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 13 23:05:22.923602 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:05:22.923602 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 630/1267] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:05:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:05:22.928569 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc7043-9678-476a-913d-9562f314d06f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:23.023578 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc7043-9678-476a-913d-9562f314d06f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:23.056997 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3190a633-2062-4272-9158-f292b5f55921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:23.057658 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3190a633-2062-4272-9158-f292b5f55921 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:23.057937 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3190a633-2062-4272-9158-f292b5f55921 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:23.058206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3190a633-2062-4272-9158-f292b5f55921 None admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:23.059031 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3190a633-2062-4272-9158-f292b5f55921 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:23.059336 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3190a633-2062-4272-9158-f292b5f55921 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:23.060787 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3190a633-2062-4272-9158-f292b5f55921 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:23.069398 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:23.078550 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 638/1268] 10.222.0.60 () {56 vars in 1143 bytes} [Wed Jul 13 23:05:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:05:23.084292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e156f1-08aa-4648-a415-cb45819ea3e2 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:23.185498 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e156f1-08aa-4648-a415-cb45819ea3e2 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:23.217887 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2cc13-03f9-4928-b12c-79a7be06d336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:23.218669 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2cc13-03f9-4928-b12c-79a7be06d336 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:23.218951 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2cc13-03f9-4928-b12c-79a7be06d336 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:23.219224 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2cc13-03f9-4928-b12c-79a7be06d336 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:23.229597 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db2cc13-03f9-4928-b12c-79a7be06d336 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:23.231781 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db2cc13-03f9-4928-b12c-79a7be06d336 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:23.248526 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-9db2cc13-03f9-4928-b12c-79a7be06d336 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:05:23.248526 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:05:23.248526 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:05:23.248526 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:05:23.248526 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:05:23.248526 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:05:23.248526 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:05:23.248526 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:05:23.248526 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:05:23.248526 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:05:23.248526 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:05:23.248526 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:05:23.248526 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:05:23.248526 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:05:23.248526 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:05:23.249553 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:05:23.249553 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:05:23.249553 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:05:23.249553 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:05:23.249553 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:05:23.249553 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:05:23.249553 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:05:23.249553 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:05:23.249553 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:05:23.249553 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:05:23.249553 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:05:23.249553 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:05:23.249553 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:05:23.249553 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:05:23.249553 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:05:23.249553 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:05:23.249553 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:05:23.250603 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:05:23.250603 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:05:23.250603 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:05:23.250603 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:05:23.250603 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 631/1269] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:05:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:05:23.255615 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3190a633-2062-4272-9158-f292b5f55921 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:23.351081 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3190a633-2062-4272-9158-f292b5f55921 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:23.383277 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5791c6-2caa-4bb2-95ea-739994d8eb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:23.383952 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5791c6-2caa-4bb2-95ea-739994d8eb03 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:23.384229 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5791c6-2caa-4bb2-95ea-739994d8eb03 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:23.384498 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5791c6-2caa-4bb2-95ea-739994d8eb03 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:23.385206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5791c6-2caa-4bb2-95ea-739994d8eb03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:23.385502 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5791c6-2caa-4bb2-95ea-739994d8eb03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:23.386850 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5791c6-2caa-4bb2-95ea-739994d8eb03 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:23.398080 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 639/1270] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:05:23.404246 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2cc13-03f9-4928-b12c-79a7be06d336 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:23.434341 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:23.500894 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2cc13-03f9-4928-b12c-79a7be06d336 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:23.533954 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83fa5e1-6931-4b8f-9945-1541c55a6493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:23.534738 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83fa5e1-6931-4b8f-9945-1541c55a6493 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:23.535017 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83fa5e1-6931-4b8f-9945-1541c55a6493 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:23.535287 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83fa5e1-6931-4b8f-9945-1541c55a6493 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:23.541852 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83fa5e1-6931-4b8f-9945-1541c55a6493 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:23.543336 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83fa5e1-6931-4b8f-9945-1541c55a6493 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:23.557361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-f83fa5e1-6931-4b8f-9945-1541c55a6493 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:05:23.557361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:05:23.557361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:05:23.557361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:05:23.557361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:05:23.557361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:05:23.557361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:05:23.557361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:05:23.557361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:05:23.557361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:05:23.557361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:05:23.557361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:05:23.557361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 23:05:23.557361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 23:05:23.557361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 23:05:23.558438 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 23:05:23.558438 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:05:23.558438 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:05:23.558438 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:05:23.558438 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:05:23.558438 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:05:23.558438 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:05:23.558438 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:05:23.558438 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:05:23.558438 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:05:23.558438 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:05:23.558438 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:05:23.558438 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:05:23.558438 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:05:23.558438 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:05:23.558438 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:05:23.558438 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:05:23.559822 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 23:05:23.559822 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 23:05:23.559822 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 23:05:23.559822 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 23:05:23.559822 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 23:05:23.559822 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 23:05:23.559822 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:05:23.559822 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:05:23.559822 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 632/1271] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:05:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:05:23.564646 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5791c6-2caa-4bb2-95ea-739994d8eb03 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:23.620521 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:23.663991 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5791c6-2caa-4bb2-95ea-739994d8eb03 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:23.696365 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092b9ca-04a5-4e57-82f3-9b06aefa17ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:23.697009 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092b9ca-04a5-4e57-82f3-9b06aefa17ba None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:23.697286 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092b9ca-04a5-4e57-82f3-9b06aefa17ba None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:23.697553 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092b9ca-04a5-4e57-82f3-9b06aefa17ba None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:23.698327 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8092b9ca-04a5-4e57-82f3-9b06aefa17ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:23.698627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8092b9ca-04a5-4e57-82f3-9b06aefa17ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:23.700684 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8092b9ca-04a5-4e57-82f3-9b06aefa17ba None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:23.712278 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 640/1272] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jul 13 23:05:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:05:23.718387 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83fa5e1-6931-4b8f-9945-1541c55a6493 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:23.805234 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:23.814423 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83fa5e1-6931-4b8f-9945-1541c55a6493 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:23.846704 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289505d7-47a0-4907-b5db-64bc79b6fbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:23.847610 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289505d7-47a0-4907-b5db-64bc79b6fbed None admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:23.847893 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289505d7-47a0-4907-b5db-64bc79b6fbed None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:23.848167 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289505d7-47a0-4907-b5db-64bc79b6fbed None admin] PATH_INFO: `/v3/projects/ab0a5581e4e444c293815e1304e20c05/users/64824ba84a024930bbc177f8a8d9a046/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:23.850095 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289505d7-47a0-4907-b5db-64bc79b6fbed None admin] RBAC: Authorizing `identity:create_grant(project_id=ab0a5581e4e444c293815e1304e20c05, user_id=64824ba84a024930bbc177f8a8d9a046, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:23.851546 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289505d7-47a0-4907-b5db-64bc79b6fbed None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:23.863635 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 633/1273] 10.222.0.60 () {58 vars in 1439 bytes} [Wed Jul 13 23:05:23 2022] PUT /identity/v3/projects/ab0a5581e4e444c293815e1304e20c05/users/64824ba84a024930bbc177f8a8d9a046/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:05:25.893503 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb5a48-ea6e-4533-b009-400db868f986 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:25.894078 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb5a48-ea6e-4533-b009-400db868f986 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:25.894078 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb5a48-ea6e-4533-b009-400db868f986 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:25.895432 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 641/1274] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:05:25.900576 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed56a3f-30db-49a0-be6b-4366c077b59c None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:25.901024 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed56a3f-30db-49a0-be6b-4366c077b59c None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:25.901126 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed56a3f-30db-49a0-be6b-4366c077b59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:25.917391 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-bed56a3f-30db-49a0-be6b-4366c077b59c None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:05:25.918562 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-bed56a3f-30db-49a0-be6b-4366c077b59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:05:25.982363 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 634/1275] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:05:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:05:26.006116 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bb5a48-ea6e-4533-b009-400db868f986 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:26.106850 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bb5a48-ea6e-4533-b009-400db868f986 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:26.126099 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:26.144213 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3846d8d2-30aa-42d0-bc7f-341b9de44fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:26.144918 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3846d8d2-30aa-42d0-bc7f-341b9de44fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:26.145198 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3846d8d2-30aa-42d0-bc7f-341b9de44fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:26.145470 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3846d8d2-30aa-42d0-bc7f-341b9de44fa5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:26.156096 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3846d8d2-30aa-42d0-bc7f-341b9de44fa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 23:05:26.163658 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-3846d8d2-30aa-42d0-bc7f-341b9de44fa5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:05:26.163658 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:05:26.163658 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:05:26.163658 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:05:26.163658 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:05:26.163658 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:05:26.163658 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:05:26.163658 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:05:26.163658 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:05:26.163658 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:05:26.163658 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:05:26.163658 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:05:26.163658 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 23:05:26.163658 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 23:05:26.163658 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 23:05:26.163658 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 23:05:26.164765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 23:05:26.164765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 23:05:26.164765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 23:05:26.164765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 23:05:26.164765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:05:26.164765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:05:26.164765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:05:26.164765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:05:26.164765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:05:26.164765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:05:26.164765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:05:26.164765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:05:26.164765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:05:26.164765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:05:26.164765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:05:26.164765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:05:26.165645 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:05:26.165645 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:05:26.165645 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:05:26.165645 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:05:26.165645 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 23:05:26.165645 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 23:05:26.165645 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 23:05:26.165645 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 23:05:26.165645 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 23:05:26.165645 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 23:05:26.165645 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 23:05:26.165645 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:05:26.165645 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 642/1276] 10.222.0.60 () {56 vars in 1096 bytes} [Wed Jul 13 23:05:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:05:26.170852 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed56a3f-30db-49a0-be6b-4366c077b59c None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:26.249944 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:26.268319 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed56a3f-30db-49a0-be6b-4366c077b59c None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:26.301047 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf5e79-affb-4642-b5af-522907d0def8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:26.301709 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf5e79-affb-4642-b5af-522907d0def8 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:26.301993 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf5e79-affb-4642-b5af-522907d0def8 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:26.302268 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf5e79-affb-4642-b5af-522907d0def8 None admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:26.303012 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cf5e79-affb-4642-b5af-522907d0def8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:26.303318 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cf5e79-affb-4642-b5af-522907d0def8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:26.304724 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cf5e79-affb-4642-b5af-522907d0def8 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:26.313162 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 635/1277] 10.222.0.60 () {56 vars in 1099 bytes} [Wed Jul 13 23:05:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:05:26.319075 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3846d8d2-30aa-42d0-bc7f-341b9de44fa5 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:26.422722 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3846d8d2-30aa-42d0-bc7f-341b9de44fa5 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:26.455320 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048b8aac-fb9e-4d88-83ee-ebf93c9b6ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:26.456128 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b8aac-fb9e-4d88-83ee-ebf93c9b6ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:26.456410 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b8aac-fb9e-4d88-83ee-ebf93c9b6ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:26.456684 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b8aac-fb9e-4d88-83ee-ebf93c9b6ee2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:26.466896 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048b8aac-fb9e-4d88-83ee-ebf93c9b6ee2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:26.468384 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048b8aac-fb9e-4d88-83ee-ebf93c9b6ee2 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:26.485509 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-048b8aac-fb9e-4d88-83ee-ebf93c9b6ee2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:05:26.485509 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:05:26.485509 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:05:26.485509 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:05:26.485509 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:05:26.485509 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:05:26.485509 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:05:26.485509 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:05:26.485509 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:05:26.485509 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:05:26.485509 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:05:26.485509 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:05:26.485509 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:05:26.485509 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:05:26.485509 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:05:26.486601 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:05:26.486601 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:05:26.486601 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:05:26.486601 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:05:26.486601 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:05:26.486601 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:05:26.486601 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:05:26.486601 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:05:26.486601 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:05:26.486601 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:05:26.486601 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:05:26.486601 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:05:26.486601 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:05:26.486601 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:05:26.486601 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:05:26.486601 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:05:26.486601 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:05:26.487561 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:05:26.487561 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:05:26.487561 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:05:26.487561 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:05:26.487561 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 643/1278] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:05:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:05:26.492500 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf5e79-affb-4642-b5af-522907d0def8 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:26.589194 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf5e79-affb-4642-b5af-522907d0def8 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:26.622291 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0d31a-5d23-4375-80c9-17ddcd065b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:26.622970 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0d31a-5d23-4375-80c9-17ddcd065b56 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:26.623253 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0d31a-5d23-4375-80c9-17ddcd065b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:26.623525 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0d31a-5d23-4375-80c9-17ddcd065b56 None admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:26.624275 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa0d31a-5d23-4375-80c9-17ddcd065b56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:26.624589 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa0d31a-5d23-4375-80c9-17ddcd065b56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:26.625997 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa0d31a-5d23-4375-80c9-17ddcd065b56 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:26.637521 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 636/1279] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:05:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:05:26.643202 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048b8aac-fb9e-4d88-83ee-ebf93c9b6ee2 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:26.674904 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:26.743486 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048b8aac-fb9e-4d88-83ee-ebf93c9b6ee2 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:26.775436 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b61f8-c4c3-47c4-b6b1-fa8e3d2f8dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:26.776257 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b61f8-c4c3-47c4-b6b1-fa8e3d2f8dca None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:26.776538 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b61f8-c4c3-47c4-b6b1-fa8e3d2f8dca None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:26.776810 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b61f8-c4c3-47c4-b6b1-fa8e3d2f8dca None admin] PATH_INFO: `/v3/users/placement` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:26.784581 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b61f8-c4c3-47c4-b6b1-fa8e3d2f8dca None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:26.786021 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b61f8-c4c3-47c4-b6b1-fa8e3d2f8dca None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:26.800404 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-f60b61f8-c4c3-47c4-b6b1-fa8e3d2f8dca None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 13 23:05:26.800404 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:05:26.800404 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:05:26.800404 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:05:26.800404 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:05:26.800404 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:05:26.800404 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:05:26.800404 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:05:26.800404 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:05:26.800404 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:05:26.800404 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:05:26.800404 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:05:26.800404 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 23:05:26.800404 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 23:05:26.800404 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 23:05:26.800404 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 23:05:26.801478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:05:26.801478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:05:26.801478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 23:05:26.801478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:05:26.801478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 23:05:26.801478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 23:05:26.801478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:05:26.801478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:05:26.801478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:05:26.801478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:05:26.801478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:05:26.801478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:05:26.801478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:05:26.801478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:05:26.801478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:05:26.801478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:05:26.801478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:05:26.801478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:05:26.802459 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:05:26.802459 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:05:26.802459 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 23:05:26.802459 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 23:05:26.802459 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 23:05:26.802459 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 23:05:26.802459 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 23:05:26.802459 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 23:05:26.802459 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 13 23:05:26.802459 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:05:26.802459 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 644/1280] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:05:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:05:26.807356 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0d31a-5d23-4375-80c9-17ddcd065b56 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:26.811118 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:26.903617 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0d31a-5d23-4375-80c9-17ddcd065b56 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:26.936756 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3e5f8-8c9a-4c5e-b900-355615e0b7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:26.937335 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3e5f8-8c9a-4c5e-b900-355615e0b7da None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:26.937619 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3e5f8-8c9a-4c5e-b900-355615e0b7da None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:26.937896 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3e5f8-8c9a-4c5e-b900-355615e0b7da None admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:26.938732 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade3e5f8-8c9a-4c5e-b900-355615e0b7da None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:26.939292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade3e5f8-8c9a-4c5e-b900-355615e0b7da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:26.940767 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade3e5f8-8c9a-4c5e-b900-355615e0b7da None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:26.957333 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 637/1281] 10.222.0.60 () {56 vars in 1143 bytes} [Wed Jul 13 23:05:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:05:26.963113 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b61f8-c4c3-47c4-b6b1-fa8e3d2f8dca None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:27.051006 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:27.060710 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b61f8-c4c3-47c4-b6b1-fa8e3d2f8dca None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:27.096606 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57260b5-5035-4b84-9f3f-aed869bb5a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:27.097654 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57260b5-5035-4b84-9f3f-aed869bb5a1f None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:27.098014 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57260b5-5035-4b84-9f3f-aed869bb5a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:27.098565 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57260b5-5035-4b84-9f3f-aed869bb5a1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:27.172073 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57260b5-5035-4b84-9f3f-aed869bb5a1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:27.173524 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57260b5-5035-4b84-9f3f-aed869bb5a1f None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:27.259237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:27.270412 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 645/1282] 10.222.0.60 () {58 vars in 1282 bytes} [Wed Jul 13 23:05:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 23:05:27.276350 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3e5f8-8c9a-4c5e-b900-355615e0b7da None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:27.340988 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:27.381941 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3e5f8-8c9a-4c5e-b900-355615e0b7da None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:27.414986 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e0636b-315c-4acc-ba42-e4c72d55e048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:27.415799 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0636b-315c-4acc-ba42-e4c72d55e048 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:27.416079 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0636b-315c-4acc-ba42-e4c72d55e048 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:27.416350 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0636b-315c-4acc-ba42-e4c72d55e048 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:27.426139 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e0636b-315c-4acc-ba42-e4c72d55e048 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:27.427613 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e0636b-315c-4acc-ba42-e4c72d55e048 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:27.445591 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-b5e0636b-315c-4acc-ba42-e4c72d55e048 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:05:27.445591 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:05:27.445591 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:05:27.445591 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:05:27.445591 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:05:27.445591 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:05:27.445591 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:05:27.445591 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:05:27.445591 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:05:27.445591 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:05:27.445591 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:05:27.445591 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:05:27.445591 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 23:05:27.445591 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 23:05:27.445591 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 23:05:27.446738 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 23:05:27.446738 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:05:27.446738 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:05:27.446738 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:05:27.446738 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:05:27.446738 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:05:27.446738 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:05:27.446738 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:05:27.446738 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:05:27.446738 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:05:27.446738 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:05:27.446738 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:05:27.446738 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:05:27.446738 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:05:27.446738 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:05:27.446738 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:05:27.446738 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:05:27.447732 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 23:05:27.447732 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 23:05:27.447732 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 23:05:27.447732 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:05:27.447732 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 638/1283] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:05:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:05:27.453104 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57260b5-5035-4b84-9f3f-aed869bb5a1f None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:27.557299 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57260b5-5035-4b84-9f3f-aed869bb5a1f None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:27.593957 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923688b4-9310-49c2-bc1b-823599c30456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:27.594627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923688b4-9310-49c2-bc1b-823599c30456 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:27.594912 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923688b4-9310-49c2-bc1b-823599c30456 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:27.595188 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923688b4-9310-49c2-bc1b-823599c30456 None admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:27.595955 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923688b4-9310-49c2-bc1b-823599c30456 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:27.596266 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923688b4-9310-49c2-bc1b-823599c30456 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:27.597663 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923688b4-9310-49c2-bc1b-823599c30456 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:27.610425 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 646/1284] 10.222.0.60 () {56 vars in 1109 bytes} [Wed Jul 13 23:05:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:05:27.616213 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e0636b-315c-4acc-ba42-e4c72d55e048 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:27.718403 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e0636b-315c-4acc-ba42-e4c72d55e048 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:27.732673 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:27.752645 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfd79a1-7892-4e89-b646-31b02b842130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:27.753433 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd79a1-7892-4e89-b646-31b02b842130 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:27.753713 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd79a1-7892-4e89-b646-31b02b842130 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:27.753986 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd79a1-7892-4e89-b646-31b02b842130 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:27.760416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfd79a1-7892-4e89-b646-31b02b842130 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:27.761889 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfd79a1-7892-4e89-b646-31b02b842130 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:27.775214 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application [None req-0dfd79a1-7892-4e89-b646-31b02b842130 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:05:27.775214 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:05:27.775214 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:05:27.775214 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:05:27.775214 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:05:27.775214 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:05:27.775214 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:05:27.775214 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:05:27.775214 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:05:27.775214 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:05:27.775214 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:05:27.775214 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:05:27.775214 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 23:05:27.775214 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 23:05:27.775214 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 23:05:27.776301 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 23:05:27.776301 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:05:27.776301 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:05:27.776301 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:05:27.776301 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:05:27.776301 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:05:27.776301 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:05:27.776301 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:05:27.776301 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application with Lock( Jul 13 23:05:27.776301 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:05:27.776301 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:05:27.776301 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:05:27.776301 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:05:27.776301 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:05:27.776301 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:05:27.776301 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:05:27.776301 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:05:27.777449 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 23:05:27.777449 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 23:05:27.777449 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 23:05:27.777449 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 23:05:27.777449 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 23:05:27.777449 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 23:05:27.777449 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 23:05:27.777449 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: ERROR keystone.server.flask.application Jul 13 23:05:27.777449 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 639/1285] 10.222.0.60 () {56 vars in 1111 bytes} [Wed Jul 13 23:05:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:05:27.782376 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923688b4-9310-49c2-bc1b-823599c30456 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:27.830606 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:27.879233 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923688b4-9310-49c2-bc1b-823599c30456 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:27.910838 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c112597c-c03b-41cf-a2ae-4b408b48802c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:27.911485 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c112597c-c03b-41cf-a2ae-4b408b48802c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:27.911791 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c112597c-c03b-41cf-a2ae-4b408b48802c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:27.912065 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c112597c-c03b-41cf-a2ae-4b408b48802c None admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:27.912838 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c112597c-c03b-41cf-a2ae-4b408b48802c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:27.913143 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c112597c-c03b-41cf-a2ae-4b408b48802c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:27.914562 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c112597c-c03b-41cf-a2ae-4b408b48802c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:27.926608 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 647/1286] 10.222.0.60 () {56 vars in 1148 bytes} [Wed Jul 13 23:05:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:05:27.932776 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfd79a1-7892-4e89-b646-31b02b842130 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:28.030325 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfd79a1-7892-4e89-b646-31b02b842130 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:28.063204 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed276f-7f16-4bf9-b6d7-104c1115e5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:28.064147 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed276f-7f16-4bf9-b6d7-104c1115e5c3 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:28.064431 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed276f-7f16-4bf9-b6d7-104c1115e5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:28.064707 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed276f-7f16-4bf9-b6d7-104c1115e5c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:28.065541 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ed276f-7f16-4bf9-b6d7-104c1115e5c3 None admin] RBAC: Adding query filter params (role.id=8cfd0b6e71e14e0983c3eb4d7478e027, scope.project.id=ab0a5581e4e444c293815e1304e20c05, user.id=64824ba84a024930bbc177f8a8d9a046) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:28.065851 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ed276f-7f16-4bf9-b6d7-104c1115e5c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:28.067300 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ed276f-7f16-4bf9-b6d7-104c1115e5c3 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:28.073121 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 640/1287] 10.222.0.60 () {56 vars in 1374 bytes} [Wed Jul 13 23:05:27 2022] GET /identity/v3/role_assignments?user.id=64824ba84a024930bbc177f8a8d9a046&scope.project.id=ab0a5581e4e444c293815e1304e20c05&role.id=8cfd0b6e71e14e0983c3eb4d7478e027 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:05:30.097669 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c16c5e-c431-49b9-8b5b-ce75dd394a05 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:30.098430 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c16c5e-c431-49b9-8b5b-ce75dd394a05 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:30.098430 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c16c5e-c431-49b9-8b5b-ce75dd394a05 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:30.099602 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 648/1288] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:05:30.105196 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f2b34-da9f-49ed-801c-b8eb938579ff None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:30.105703 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f2b34-da9f-49ed-801c-b8eb938579ff None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:30.105942 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f2b34-da9f-49ed-801c-b8eb938579ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:30.126824 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-3d7f2b34-da9f-49ed-801c-b8eb938579ff None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:05:30.128143 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-3d7f2b34-da9f-49ed-801c-b8eb938579ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:05:30.165929 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:30.207060 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 641/1289] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:05:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:05:30.229616 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c16c5e-c431-49b9-8b5b-ce75dd394a05 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:30.334096 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c16c5e-c431-49b9-8b5b-ce75dd394a05 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:30.345403 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:30.369832 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1661a877-b4e3-4a4d-a9cd-bb5f5e16dda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:30.370631 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1661a877-b4e3-4a4d-a9cd-bb5f5e16dda9 None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:30.370913 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1661a877-b4e3-4a4d-a9cd-bb5f5e16dda9 None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:30.371715 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1661a877-b4e3-4a4d-a9cd-bb5f5e16dda9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:30.372364 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1661a877-b4e3-4a4d-a9cd-bb5f5e16dda9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:30.373741 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1661a877-b4e3-4a4d-a9cd-bb5f5e16dda9 None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:30.385617 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-1661a877-b4e3-4a4d-a9cd-bb5f5e16dda9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 23:05:30.385617 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:05:30.385617 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 23:05:30.385617 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 23:05:30.385617 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 23:05:30.385617 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 23:05:30.385617 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 23:05:30.385617 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:05:30.385617 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 23:05:30.385617 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:05:30.385617 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 23:05:30.385617 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:05:30.385617 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:05:30.385617 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:05:30.385617 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:05:30.385617 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:05:30.385617 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:05:30.387965 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:05:30.387965 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:05:30.387965 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:05:30.387965 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:05:30.387965 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:05:30.387965 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:05:30.387965 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 23:05:30.387965 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 23:05:30.387965 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 23:05:30.387965 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 23:05:30.387965 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:05:30.387965 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:05:30.387965 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:05:30.387965 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:05:30.387965 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:05:30.387965 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:05:30.388992 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:05:30.388992 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:05:30.388992 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:05:30.388992 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:05:30.388992 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:05:30.388992 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:05:30.388992 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:05:30.388992 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:05:30.388992 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:05:30.388992 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:05:30.388992 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 23:05:30.388992 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:05:30.388992 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 23:05:30.388992 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:05:30.388992 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 649/1290] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Jul 13 23:05:30 2022] GET /identity/v3/services/placement => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:05:30.393909 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f2b34-da9f-49ed-801c-b8eb938579ff None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:30.499287 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f2b34-da9f-49ed-801c-b8eb938579ff None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:30.533932 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c1c93-5ca8-4433-9301-c79a78bebaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:30.534589 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c1c93-5ca8-4433-9301-c79a78bebaa6 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:30.534874 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c1c93-5ca8-4433-9301-c79a78bebaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:30.535151 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c1c93-5ca8-4433-9301-c79a78bebaa6 None admin] PATH_INFO: `/v3/services` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:30.535895 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c1c93-5ca8-4433-9301-c79a78bebaa6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:30.536202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c1c93-5ca8-4433-9301-c79a78bebaa6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:30.537598 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c1c93-5ca8-4433-9301-c79a78bebaa6 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:30.542498 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 642/1291] 10.222.0.60 () {56 vars in 1116 bytes} [Wed Jul 13 23:05:30 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:05:30.550934 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1661a877-b4e3-4a4d-a9cd-bb5f5e16dda9 None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:30.650517 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1661a877-b4e3-4a4d-a9cd-bb5f5e16dda9 None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:30.682737 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f619c32-0e46-408f-a71a-70078a83d7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:30.683386 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f619c32-0e46-408f-a71a-70078a83d7fd None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:30.684195 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f619c32-0e46-408f-a71a-70078a83d7fd None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:30.684466 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f619c32-0e46-408f-a71a-70078a83d7fd None admin] PATH_INFO: `/v3/services` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:30.685169 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f619c32-0e46-408f-a71a-70078a83d7fd None admin] RBAC: Adding query filter params (type=placement) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:30.685468 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f619c32-0e46-408f-a71a-70078a83d7fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:30.686833 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f619c32-0e46-408f-a71a-70078a83d7fd None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:30.691470 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 650/1292] 10.222.0.60 () {56 vars in 1116 bytes} [Wed Jul 13 23:05:30 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:05:32.717457 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c7705-90e4-4dfb-a189-3ca81c78f40b None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:32.718014 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c7705-90e4-4dfb-a189-3ca81c78f40b None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:32.718014 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c7705-90e4-4dfb-a189-3ca81c78f40b None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:32.719385 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 643/1293] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:05:32.724504 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34cf1a-0829-45c8-a116-a340f93adef4 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:32.724912 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34cf1a-0829-45c8-a116-a340f93adef4 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:32.725073 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34cf1a-0829-45c8-a116-a340f93adef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:32.742094 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-3a34cf1a-0829-45c8-a116-a340f93adef4 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:05:32.743290 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3a34cf1a-0829-45c8-a116-a340f93adef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:05:32.771812 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:32.809515 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 651/1294] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:05:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:05:32.832607 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c7705-90e4-4dfb-a189-3ca81c78f40b None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:32.864222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:32.933910 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c7705-90e4-4dfb-a189-3ca81c78f40b None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:32.967702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c77c6-2bfb-4ed8-85e8-36a445af2637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:32.968278 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c77c6-2bfb-4ed8-85e8-36a445af2637 None admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:32.968560 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c77c6-2bfb-4ed8-85e8-36a445af2637 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:32.968831 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c77c6-2bfb-4ed8-85e8-36a445af2637 None admin] PATH_INFO: `/v3/services` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:32.969541 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2c77c6-2bfb-4ed8-85e8-36a445af2637 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:32.970921 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2c77c6-2bfb-4ed8-85e8-36a445af2637 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:32.985738 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 644/1295] 10.222.0.60 () {60 vars in 1141 bytes} [Wed Jul 13 23:05:32 2022] POST /identity/v3/services => generated 246 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:05:35.058242 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26db1a-9519-4b4b-a7ba-cf2e3693077a None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:35.059313 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26db1a-9519-4b4b-a7ba-cf2e3693077a None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:35.059313 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26db1a-9519-4b4b-a7ba-cf2e3693077a None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:35.060704 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 652/1296] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:05:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:05:35.066198 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f432a1-4159-438f-a6ff-3495068ef15f None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:35.066627 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f432a1-4159-438f-a6ff-3495068ef15f None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:35.066758 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f432a1-4159-438f-a6ff-3495068ef15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:35.083608 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-97f432a1-4159-438f-a6ff-3495068ef15f None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:05:35.084804 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-97f432a1-4159-438f-a6ff-3495068ef15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:05:35.161570 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 645/1297] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:05:35 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:05:35.184800 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26db1a-9519-4b4b-a7ba-cf2e3693077a None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:35.285536 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26db1a-9519-4b4b-a7ba-cf2e3693077a None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:35.319673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e9f370-87e3-461c-878a-d7ed14cda85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:35.320388 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9f370-87e3-461c-878a-d7ed14cda85c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:35.320669 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9f370-87e3-461c-878a-d7ed14cda85c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:35.320943 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9f370-87e3-461c-878a-d7ed14cda85c None admin] PATH_INFO: `/v3/services/placement` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:35.321591 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e9f370-87e3-461c-878a-d7ed14cda85c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:35.322971 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e9f370-87e3-461c-878a-d7ed14cda85c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:35.334616 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-f9e9f370-87e3-461c-878a-d7ed14cda85c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 23:05:35.334616 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:05:35.334616 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 23:05:35.334616 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 23:05:35.334616 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 23:05:35.334616 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 23:05:35.334616 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 23:05:35.334616 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:05:35.334616 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 23:05:35.334616 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:05:35.334616 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 23:05:35.334616 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:05:35.334616 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:05:35.334616 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:05:35.334616 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:05:35.334616 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:05:35.334616 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:05:35.335889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:05:35.335889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:05:35.335889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:05:35.335889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:05:35.335889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:05:35.335889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:05:35.335889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 23:05:35.335889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 23:05:35.335889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 23:05:35.335889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 23:05:35.335889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:05:35.335889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:05:35.335889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:05:35.335889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:05:35.335889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:05:35.335889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:05:35.337010 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:05:35.337010 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:05:35.337010 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:05:35.337010 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:05:35.337010 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:05:35.337010 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:05:35.337010 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:05:35.337010 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:05:35.337010 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:05:35.337010 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:05:35.337010 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 23:05:35.337010 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:05:35.337010 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 23:05:35.337010 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:05:35.337010 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 653/1298] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Jul 13 23:05:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:05:35.342102 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f432a1-4159-438f-a6ff-3495068ef15f None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:35.360715 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:35.439783 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f432a1-4159-438f-a6ff-3495068ef15f None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:35.471538 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db364f3-52bf-405c-9b33-5f57ec6c0b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:35.472214 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db364f3-52bf-405c-9b33-5f57ec6c0b73 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:35.472492 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db364f3-52bf-405c-9b33-5f57ec6c0b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:35.472763 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db364f3-52bf-405c-9b33-5f57ec6c0b73 None admin] PATH_INFO: `/v3/services` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:35.473462 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db364f3-52bf-405c-9b33-5f57ec6c0b73 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:35.473770 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db364f3-52bf-405c-9b33-5f57ec6c0b73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:35.475147 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db364f3-52bf-405c-9b33-5f57ec6c0b73 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:35.480726 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 646/1299] 10.222.0.60 () {56 vars in 1116 bytes} [Wed Jul 13 23:05:35 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:05:35.486635 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e9f370-87e3-461c-878a-d7ed14cda85c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:35.517814 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:35.585522 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e9f370-87e3-461c-878a-d7ed14cda85c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:35.618162 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8175af-4c04-40bc-bd60-039712387bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:35.618813 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8175af-4c04-40bc-bd60-039712387bdd None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:35.619096 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8175af-4c04-40bc-bd60-039712387bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:35.619374 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8175af-4c04-40bc-bd60-039712387bdd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:35.620195 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8175af-4c04-40bc-bd60-039712387bdd None admin] RBAC: Adding query filter params (interface=public, service_id=2083be8a4ead49fb93bc3d6edda22ede, region_id=RegionOne) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:35.620507 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8175af-4c04-40bc-bd60-039712387bdd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:35.622524 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8175af-4c04-40bc-bd60-039712387bdd None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:35.627472 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 654/1300] 10.222.0.60 () {56 vars in 1251 bytes} [Wed Jul 13 23:05:35 2022] GET /identity/v3/endpoints?service_id=2083be8a4ead49fb93bc3d6edda22ede&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:05:35.633115 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db364f3-52bf-405c-9b33-5f57ec6c0b73 None admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:35.711755 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:35.729513 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db364f3-52bf-405c-9b33-5f57ec6c0b73 None admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:35.763480 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5be003-b5b4-4a1d-8035-b93d28df165e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:35.764175 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5be003-b5b4-4a1d-8035-b93d28df165e None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:35.764458 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5be003-b5b4-4a1d-8035-b93d28df165e None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:35.764733 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5be003-b5b4-4a1d-8035-b93d28df165e None admin] PATH_INFO: `/v3/services` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:35.765416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5be003-b5b4-4a1d-8035-b93d28df165e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:35.766807 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5be003-b5b4-4a1d-8035-b93d28df165e None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:35.773730 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 647/1301] 10.222.0.60 () {56 vars in 1087 bytes} [Wed Jul 13 23:05:35 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:05:37.797469 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47144ae9-4981-463c-bc68-3f28333a6dbc None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:37.798036 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47144ae9-4981-463c-bc68-3f28333a6dbc None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:37.798036 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47144ae9-4981-463c-bc68-3f28333a6dbc None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:37.799381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 655/1302] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:05:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:05:37.804539 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d7654-f028-48e1-a59e-7f5953ffe374 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:37.804930 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d7654-f028-48e1-a59e-7f5953ffe374 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:37.805088 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d7654-f028-48e1-a59e-7f5953ffe374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:37.821712 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-6b1d7654-f028-48e1-a59e-7f5953ffe374 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:05:37.822908 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-6b1d7654-f028-48e1-a59e-7f5953ffe374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:05:37.888871 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 648/1303] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:05:37 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 23:05:37.911401 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47144ae9-4981-463c-bc68-3f28333a6dbc None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:38.006485 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:38.014208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47144ae9-4981-463c-bc68-3f28333a6dbc None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:38.046421 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae283b-b788-42f9-ac92-faa287960d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:38.047210 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae283b-b788-42f9-ac92-faa287960d3c None admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:38.047562 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae283b-b788-42f9-ac92-faa287960d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:38.047780 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae283b-b788-42f9-ac92-faa287960d3c None admin] PATH_INFO: `/v3/services/placement` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:38.048421 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caae283b-b788-42f9-ac92-faa287960d3c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:38.049801 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caae283b-b788-42f9-ac92-faa287960d3c None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:38.061953 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application [None req-caae283b-b788-42f9-ac92-faa287960d3c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 23:05:38.061953 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:05:38.061953 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 23:05:38.061953 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 23:05:38.061953 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 23:05:38.061953 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 23:05:38.061953 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 23:05:38.061953 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:05:38.061953 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 23:05:38.061953 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:05:38.061953 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 23:05:38.061953 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:05:38.061953 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 23:05:38.061953 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 23:05:38.061953 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 23:05:38.061953 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 23:05:38.061953 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 23:05:38.062966 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 23:05:38.062966 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 23:05:38.062966 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 23:05:38.062966 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 23:05:38.062966 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 23:05:38.062966 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 23:05:38.062966 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 23:05:38.062966 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 23:05:38.062966 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 23:05:38.062966 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 23:05:38.062966 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 23:05:38.062966 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 23:05:38.062966 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 23:05:38.062966 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 23:05:38.062966 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 23:05:38.062966 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 23:05:38.064172 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 23:05:38.064172 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application with Lock( Jul 13 23:05:38.064172 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 23:05:38.064172 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self._enter() Jul 13 23:05:38.064172 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 23:05:38.064172 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 23:05:38.064172 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 23:05:38.064172 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application return self.creator() Jul 13 23:05:38.064172 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 23:05:38.064172 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application created_value = creator( Jul 13 23:05:38.064172 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 23:05:38.064172 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 23:05:38.064172 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 23:05:38.064172 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: ERROR keystone.server.flask.application Jul 13 23:05:38.064172 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 656/1304] 10.222.0.60 () {56 vars in 1117 bytes} [Wed Jul 13 23:05:37 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 23:05:38.069425 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d7654-f028-48e1-a59e-7f5953ffe374 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:38.157150 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:38.181233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d7654-f028-48e1-a59e-7f5953ffe374 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:38.215542 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c51fc1-f9d1-4db9-b9a9-9732d9d5fb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:38.216222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c51fc1-f9d1-4db9-b9a9-9732d9d5fb22 None admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:38.216504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c51fc1-f9d1-4db9-b9a9-9732d9d5fb22 None admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:38.216780 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c51fc1-f9d1-4db9-b9a9-9732d9d5fb22 None admin] PATH_INFO: `/v3/services` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:38.217483 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c51fc1-f9d1-4db9-b9a9-9732d9d5fb22 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:05:38.217789 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c51fc1-f9d1-4db9-b9a9-9732d9d5fb22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:38.219182 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c51fc1-f9d1-4db9-b9a9-9732d9d5fb22 None admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:38.224226 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 649/1305] 10.222.0.60 () {56 vars in 1116 bytes} [Wed Jul 13 23:05:38 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:05:38.230480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae283b-b788-42f9-ac92-faa287960d3c None admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:05:38.328907 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae283b-b788-42f9-ac92-faa287960d3c None admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:05:38.362621 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c5493-4e4c-43e7-ba9c-9b242642bd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:05:38.363286 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c5493-4e4c-43e7-ba9c-9b242642bd7a None admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:05:38.363598 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c5493-4e4c-43e7-ba9c-9b242642bd7a None admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:05:38.363873 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c5493-4e4c-43e7-ba9c-9b242642bd7a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:05:38.364627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72c5493-4e4c-43e7-ba9c-9b242642bd7a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:05:38.366001 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72c5493-4e4c-43e7-ba9c-9b242642bd7a None admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:05:38.385053 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 657/1306] 10.222.0.60 () {60 vars in 1144 bytes} [Wed Jul 13 23:05:38 2022] POST /identity/v3/endpoints => generated 331 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:05:48.221271 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=30 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:05:48.380901 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:06:44.639790 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9944b9b-8ccf-4c9e-9af5-26c141517d41 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:06:44.640379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9944b9b-8ccf-4c9e-9af5-26c141517d41 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:06:44.640379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9944b9b-8ccf-4c9e-9af5-26c141517d41 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:06:44.642181 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 650/1307] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:06:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:06:44.647405 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729cf37-e619-4e94-b0bc-55163709258a None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:06:44.648018 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729cf37-e619-4e94-b0bc-55163709258a None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:06:44.648018 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729cf37-e619-4e94-b0bc-55163709258a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:06:44.782681 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-4729cf37-e619-4e94-b0bc-55163709258a None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:06:44.784889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4729cf37-e619-4e94-b0bc-55163709258a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:06:44.939581 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 658/1308] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:06:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:06:45.003046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2178639d-fe3c-44cb-abce-c07c52b61c61 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:06:45.003512 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2178639d-fe3c-44cb-abce-c07c52b61c61 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:06:45.003816 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2178639d-fe3c-44cb-abce-c07c52b61c61 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:06:45.005761 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 651/1309] 10.222.0.60 () {54 vars in 964 bytes} [Wed Jul 13 23:06:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:06:45.013971 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5756e52-9a68-422c-bff4-f7c5cdef1adc None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:06:45.014282 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5756e52-9a68-422c-bff4-f7c5cdef1adc None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:06:45.014654 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5756e52-9a68-422c-bff4-f7c5cdef1adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:06:45.119894 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-f5756e52-9a68-422c-bff4-f7c5cdef1adc None None] MFA Rules not processed for user `2d670df2aff5478ebf61f55358d554de`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:06:45.121157 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f5756e52-9a68-422c-bff4-f7c5cdef1adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:06:45.155258 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:06:45.243060 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 659/1310] 10.222.0.60 () {58 vars in 1061 bytes} [Wed Jul 13 23:06:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:06:45.260930 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2178639d-fe3c-44cb-abce-c07c52b61c61 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:06:45.263311 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-2178639d-fe3c-44cb-abce-c07c52b61c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:06:45.428864 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2178639d-fe3c-44cb-abce-c07c52b61c61 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:06:45.564180 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a16e2-9a4a-44fd-bdaa-c785d8a53bb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:06:45.565063 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a16e2-9a4a-44fd-bdaa-c785d8a53bb5 service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:06:45.565433 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a16e2-9a4a-44fd-bdaa-c785d8a53bb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:06:45.565743 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a16e2-9a4a-44fd-bdaa-c785d8a53bb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:06:45.568264 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-4f2a16e2-9a4a-44fd-bdaa-c785d8a53bb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:06:45.644004 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:06:45.747823 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2a16e2-9a4a-44fd-bdaa-c785d8a53bb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:06:45.749227 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2a16e2-9a4a-44fd-bdaa-c785d8a53bb5 service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:06:45.781505 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 652/1311] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:06:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:06:48.637514 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972dc4ce-a627-403a-a3bc-54ef92c35180 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:06:48.638017 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972dc4ce-a627-403a-a3bc-54ef92c35180 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:06:48.638205 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972dc4ce-a627-403a-a3bc-54ef92c35180 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:06:48.639489 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 660/1312] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:06:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:06:48.645332 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8564d-1ccd-45d2-8b61-c7cf398abf27 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:06:48.645784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8564d-1ccd-45d2-8b61-c7cf398abf27 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:06:48.646120 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8564d-1ccd-45d2-8b61-c7cf398abf27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:06:48.724921 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-a7b8564d-1ccd-45d2-8b61-c7cf398abf27 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:06:48.726464 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-a7b8564d-1ccd-45d2-8b61-c7cf398abf27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:06:48.855944 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 653/1313] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:06:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:06:48.913491 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84ceab-d7b4-4e35-8b62-936755c44f5f None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:06:48.913776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84ceab-d7b4-4e35-8b62-936755c44f5f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:06:48.914059 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84ceab-d7b4-4e35-8b62-936755c44f5f None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:06:48.915515 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 661/1314] 10.222.0.60 () {54 vars in 964 bytes} [Wed Jul 13 23:06:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:06:48.923698 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04cad84-f297-4122-81a1-6fc4c7fb4ad9 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:06:48.923992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04cad84-f297-4122-81a1-6fc4c7fb4ad9 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:06:48.924271 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04cad84-f297-4122-81a1-6fc4c7fb4ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:06:48.935028 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:06:49.008711 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-c04cad84-f297-4122-81a1-6fc4c7fb4ad9 None None] MFA Rules not processed for user `2d670df2aff5478ebf61f55358d554de`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:06:49.009960 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-c04cad84-f297-4122-81a1-6fc4c7fb4ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:06:49.089882 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 654/1315] 10.222.0.60 () {58 vars in 1061 bytes} [Wed Jul 13 23:06:48 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:06:49.099119 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84ceab-d7b4-4e35-8b62-936755c44f5f None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:06:49.100672 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-bf84ceab-d7b4-4e35-8b62-936755c44f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:06:49.227448 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84ceab-d7b4-4e35-8b62-936755c44f5f None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:06:49.309636 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:06:49.313042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ac242-a5f0-4753-8c63-799de815350a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:06:49.313995 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ac242-a5f0-4753-8c63-799de815350a service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:06:49.314284 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ac242-a5f0-4753-8c63-799de815350a service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:06:49.314563 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ac242-a5f0-4753-8c63-799de815350a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:06:49.316315 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a91ac242-a5f0-4753-8c63-799de815350a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:06:49.451084 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ac242-a5f0-4753-8c63-799de815350a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:06:49.452571 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ac242-a5f0-4753-8c63-799de815350a service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:06:49.472684 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 662/1316] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:06:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:06:52.101445 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb452689-dc3b-4bbd-b1d5-ab5926a9e2a6 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:06:52.102014 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb452689-dc3b-4bbd-b1d5-ab5926a9e2a6 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:06:52.102014 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb452689-dc3b-4bbd-b1d5-ab5926a9e2a6 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:06:52.104024 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 655/1317] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:06:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:06:52.110820 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1f341-4bee-4860-8117-49b39bae1535 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:06:52.111268 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1f341-4bee-4860-8117-49b39bae1535 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:06:52.111729 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1f341-4bee-4860-8117-49b39bae1535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:06:52.180864 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-82c1f341-4bee-4860-8117-49b39bae1535 None None] MFA Rules not processed for user `9159a16d47ec45bd861706097b1a5a73`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:06:52.182264 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-82c1f341-4bee-4860-8117-49b39bae1535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:06:52.243485 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:06:52.295177 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 663/1318] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:06:52 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:06:52.341094 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb452689-dc3b-4bbd-b1d5-ab5926a9e2a6 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:06:52.342637 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-eb452689-dc3b-4bbd-b1d5-ab5926a9e2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:06:52.457484 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb452689-dc3b-4bbd-b1d5-ab5926a9e2a6 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:06:52.460901 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:06:52.542723 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f8080-3e9a-4de8-ba12-6b05f78f687e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:06:52.543421 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f8080-3e9a-4de8-ba12-6b05f78f687e service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:06:52.543752 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f8080-3e9a-4de8-ba12-6b05f78f687e service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:06:52.544028 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f8080-3e9a-4de8-ba12-6b05f78f687e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:06:52.545691 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-911f8080-3e9a-4de8-ba12-6b05f78f687e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:06:52.659836 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f8080-3e9a-4de8-ba12-6b05f78f687e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:06:52.661323 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f8080-3e9a-4de8-ba12-6b05f78f687e service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:06:52.680704 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 656/1319] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:06:52 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:06:55.567210 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3732338a-9f0b-49a0-86f1-02e294b2fd74 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:06:55.567801 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3732338a-9f0b-49a0-86f1-02e294b2fd74 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:06:55.567801 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3732338a-9f0b-49a0-86f1-02e294b2fd74 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:06:55.569192 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 664/1320] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:06:55.575089 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a104e-483c-4c44-a73f-9393dabb89cc None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:06:55.575590 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a104e-483c-4c44-a73f-9393dabb89cc None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:06:55.575688 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a104e-483c-4c44-a73f-9393dabb89cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:06:55.628055 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:06:55.658065 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-1d7a104e-483c-4c44-a73f-9393dabb89cc None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:06:55.659358 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-1d7a104e-483c-4c44-a73f-9393dabb89cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:06:55.772865 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 657/1321] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:06:55 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:06:55.817064 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3732338a-9f0b-49a0-86f1-02e294b2fd74 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:06:55.818632 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3732338a-9f0b-49a0-86f1-02e294b2fd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:06:55.928482 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3732338a-9f0b-49a0-86f1-02e294b2fd74 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:06:56.011064 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f920aa4-9e10-4838-92bd-d2f2084aeac1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:06:56.011784 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f920aa4-9e10-4838-92bd-d2f2084aeac1 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:06:56.012074 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f920aa4-9e10-4838-92bd-d2f2084aeac1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:06:56.012403 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f920aa4-9e10-4838-92bd-d2f2084aeac1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:06:56.014186 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1f920aa4-9e10-4838-92bd-d2f2084aeac1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:06:56.022135 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:06:56.131259 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f920aa4-9e10-4838-92bd-d2f2084aeac1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:06:56.132815 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f920aa4-9e10-4838-92bd-d2f2084aeac1 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:06:56.153078 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 665/1322] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:06:55 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:06:58.987905 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4612a-516a-44b9-a80b-19cd12ed827e None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:06:58.988447 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4612a-516a-44b9-a80b-19cd12ed827e None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:06:58.988897 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4612a-516a-44b9-a80b-19cd12ed827e None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:06:58.991485 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 658/1323] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:06:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:06:58.997244 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb89e05-56e1-4038-8903-aeef112f9ad3 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:06:58.997694 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb89e05-56e1-4038-8903-aeef112f9ad3 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:06:58.997814 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb89e05-56e1-4038-8903-aeef112f9ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:06:59.065023 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-0eb89e05-56e1-4038-8903-aeef112f9ad3 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:06:59.066252 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0eb89e05-56e1-4038-8903-aeef112f9ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:06:59.129987 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:06:59.173358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 666/1324] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:06:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:06:59.210763 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4612a-516a-44b9-a80b-19cd12ed827e None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:06:59.212489 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-7db4612a-516a-44b9-a80b-19cd12ed827e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:06:59.330692 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4612a-516a-44b9-a80b-19cd12ed827e None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:06:59.344260 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:06:59.417593 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43aa01-b681-413a-9ef4-c8f8df730a6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:06:59.418241 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43aa01-b681-413a-9ef4-c8f8df730a6d service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:06:59.418521 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43aa01-b681-413a-9ef4-c8f8df730a6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:06:59.418796 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43aa01-b681-413a-9ef4-c8f8df730a6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:06:59.420487 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-fb43aa01-b681-413a-9ef4-c8f8df730a6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:06:59.541077 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb43aa01-b681-413a-9ef4-c8f8df730a6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:06:59.542570 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb43aa01-b681-413a-9ef4-c8f8df730a6d service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:06:59.563631 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 659/1325] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:06:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:07:01.681185 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d1196-0235-4cbb-955a-0ecf1ce417ab None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:01.681724 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d1196-0235-4cbb-955a-0ecf1ce417ab None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:01.681872 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d1196-0235-4cbb-955a-0ecf1ce417ab None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:01.684223 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 667/1326] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:07:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:07:01.691008 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6774fa-0cb3-413b-9ace-0c1c9642b29b None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:01.691620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6774fa-0cb3-413b-9ace-0c1c9642b29b None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:01.691959 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6774fa-0cb3-413b-9ace-0c1c9642b29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:01.746312 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:07:01.765236 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-0a6774fa-0cb3-413b-9ace-0c1c9642b29b None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:07:01.766471 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-0a6774fa-0cb3-413b-9ace-0c1c9642b29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:01.867618 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 660/1327] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:07:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:07:01.904527 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d1196-0235-4cbb-955a-0ecf1ce417ab None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:07:01.906049 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3c2d1196-0235-4cbb-955a-0ecf1ce417ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:02.012587 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d1196-0235-4cbb-955a-0ecf1ce417ab None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:07:02.092745 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37622fa5-f7a1-47aa-8645-6199405305f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:07:02.093643 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37622fa5-f7a1-47aa-8645-6199405305f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:02.093943 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37622fa5-f7a1-47aa-8645-6199405305f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:02.094335 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37622fa5-f7a1-47aa-8645-6199405305f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:02.096692 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-37622fa5-f7a1-47aa-8645-6199405305f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:02.117637 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:07:02.221924 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37622fa5-f7a1-47aa-8645-6199405305f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:07:02.223367 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37622fa5-f7a1-47aa-8645-6199405305f0 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:07:02.241838 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 668/1328] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:07:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:07:04.399605 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75109939-9640-436c-bfaa-0cec46fcbf46 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:04.400200 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75109939-9640-436c-bfaa-0cec46fcbf46 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:04.400200 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75109939-9640-436c-bfaa-0cec46fcbf46 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:04.401585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 661/1329] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:07:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:07:04.406678 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72607ce-9c49-49d3-80f5-1669972b2ff9 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:04.407131 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72607ce-9c49-49d3-80f5-1669972b2ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:04.407323 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72607ce-9c49-49d3-80f5-1669972b2ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:04.476643 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-f72607ce-9c49-49d3-80f5-1669972b2ff9 None None] MFA Rules not processed for user `9159a16d47ec45bd861706097b1a5a73`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:07:04.477812 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f72607ce-9c49-49d3-80f5-1669972b2ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:04.545662 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:07:04.582651 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 669/1330] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:07:04 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:07:04.619864 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75109939-9640-436c-bfaa-0cec46fcbf46 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:07:04.621455 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-75109939-9640-436c-bfaa-0cec46fcbf46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:04.724822 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75109939-9640-436c-bfaa-0cec46fcbf46 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:07:04.744939 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:07:04.800615 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e062dd85-aa7f-4537-adfa-a870c1a330bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:07:04.801279 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062dd85-aa7f-4537-adfa-a870c1a330bd service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:04.801583 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062dd85-aa7f-4537-adfa-a870c1a330bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:04.801840 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062dd85-aa7f-4537-adfa-a870c1a330bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:04.803515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-e062dd85-aa7f-4537-adfa-a870c1a330bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:04.911357 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e062dd85-aa7f-4537-adfa-a870c1a330bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:07:04.912852 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e062dd85-aa7f-4537-adfa-a870c1a330bd service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:07:04.933049 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 662/1331] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:07:04 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:07:07.237079 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb510429-e5d3-4b8d-b941-978a2ddbc300 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:07.237691 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb510429-e5d3-4b8d-b941-978a2ddbc300 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:07.237691 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb510429-e5d3-4b8d-b941-978a2ddbc300 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:07.239012 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 670/1332] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:07:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:07:07.244138 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c57a89-6a80-44ed-a737-cd6cc3d0a5eb None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:07.244576 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c57a89-6a80-44ed-a737-cd6cc3d0a5eb None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:07.244672 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c57a89-6a80-44ed-a737-cd6cc3d0a5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:07.305689 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:07:07.308316 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-97c57a89-6a80-44ed-a737-cd6cc3d0a5eb None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:07:07.310179 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-97c57a89-6a80-44ed-a737-cd6cc3d0a5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:07.404445 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 663/1333] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:07:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:07:07.440955 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb510429-e5d3-4b8d-b941-978a2ddbc300 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:07:07.442550 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-cb510429-e5d3-4b8d-b941-978a2ddbc300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:07.543548 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb510429-e5d3-4b8d-b941-978a2ddbc300 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:07:07.616821 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3673db-8a68-4e7f-9f66-eb9c29948a5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:07:07.617468 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3673db-8a68-4e7f-9f66-eb9c29948a5a service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:07.617746 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3673db-8a68-4e7f-9f66-eb9c29948a5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:07.618014 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3673db-8a68-4e7f-9f66-eb9c29948a5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:07.620258 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6a3673db-8a68-4e7f-9f66-eb9c29948a5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:07.646937 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:07:07.729133 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3673db-8a68-4e7f-9f66-eb9c29948a5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:07:07.730558 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3673db-8a68-4e7f-9f66-eb9c29948a5a service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:07:07.748497 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 671/1334] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:07:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:07:09.815261 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a97b25-a057-43d7-a415-d6609730576e None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:09.815886 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a97b25-a057-43d7-a415-d6609730576e None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:09.815886 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a97b25-a057-43d7-a415-d6609730576e None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:09.817206 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 664/1335] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:07:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:07:09.822324 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d58eb0-2b69-4654-be64-cfb301a4a13c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:09.822755 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d58eb0-2b69-4654-be64-cfb301a4a13c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:09.822886 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d58eb0-2b69-4654-be64-cfb301a4a13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:09.886946 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-85d58eb0-2b69-4654-be64-cfb301a4a13c None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:07:09.888199 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-85d58eb0-2b69-4654-be64-cfb301a4a13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:09.963886 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:07:09.987179 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 672/1336] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:07:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:07:10.026862 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a97b25-a057-43d7-a415-d6609730576e None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:07:10.028424 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-19a97b25-a057-43d7-a415-d6609730576e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:10.128250 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a97b25-a057-43d7-a415-d6609730576e None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:07:10.158059 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:07:10.202610 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b584a701-9cd0-42f3-83ef-1a7346472ff1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:07:10.203259 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584a701-9cd0-42f3-83ef-1a7346472ff1 service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:10.203595 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584a701-9cd0-42f3-83ef-1a7346472ff1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:10.203849 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584a701-9cd0-42f3-83ef-1a7346472ff1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:10.205517 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-b584a701-9cd0-42f3-83ef-1a7346472ff1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:10.311269 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b584a701-9cd0-42f3-83ef-1a7346472ff1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:07:10.312993 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b584a701-9cd0-42f3-83ef-1a7346472ff1 service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:07:10.330917 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 665/1337] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:07:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:07:13.014498 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273587f-48fe-4915-a9ec-1d1644f3173b None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:13.015038 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273587f-48fe-4915-a9ec-1d1644f3173b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:13.015038 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a273587f-48fe-4915-a9ec-1d1644f3173b None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:13.016522 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 673/1338] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:07:13.022281 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24203b3-0455-4ee1-a570-e41edefd4046 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:13.022697 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24203b3-0455-4ee1-a570-e41edefd4046 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:13.022845 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24203b3-0455-4ee1-a570-e41edefd4046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:13.094075 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-a24203b3-0455-4ee1-a570-e41edefd4046 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:07:13.095298 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-a24203b3-0455-4ee1-a570-e41edefd4046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:13.105580 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:07:13.192593 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 666/1339] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:07:13 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:07:13.228889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a273587f-48fe-4915-a9ec-1d1644f3173b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:07:13.230434 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a273587f-48fe-4915-a9ec-1d1644f3173b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:13.333131 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a273587f-48fe-4915-a9ec-1d1644f3173b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:07:13.406121 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec2150-14bc-49b9-b750-ef932eef3069 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:07:13.406763 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec2150-14bc-49b9-b750-ef932eef3069 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:13.407045 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec2150-14bc-49b9-b750-ef932eef3069 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:13.407324 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec2150-14bc-49b9-b750-ef932eef3069 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:13.409008 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b4ec2150-14bc-49b9-b750-ef932eef3069 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:13.443464 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:07:13.515262 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ec2150-14bc-49b9-b750-ef932eef3069 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:07:13.516741 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ec2150-14bc-49b9-b750-ef932eef3069 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:07:13.535269 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 674/1340] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:07:13 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:07:17.909386 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc422fdf-7152-4956-91e8-b18a135f9cc9 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:17.909958 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc422fdf-7152-4956-91e8-b18a135f9cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:17.910377 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc422fdf-7152-4956-91e8-b18a135f9cc9 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:17.912941 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 667/1341] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:07:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:07:17.920485 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c551a328-4964-43d9-995b-32d3f07fd058 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:17.921329 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c551a328-4964-43d9-995b-32d3f07fd058 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:17.921329 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c551a328-4964-43d9-995b-32d3f07fd058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:18.019817 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c551a328-4964-43d9-995b-32d3f07fd058 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:07:18.021053 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c551a328-4964-43d9-995b-32d3f07fd058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:18.121713 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:07:18.135670 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 675/1342] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:07:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:07:18.176080 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc422fdf-7152-4956-91e8-b18a135f9cc9 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:07:18.177833 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-bc422fdf-7152-4956-91e8-b18a135f9cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:18.295758 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc422fdf-7152-4956-91e8-b18a135f9cc9 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:07:18.339848 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:07:18.384774 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2d088-980f-4f01-9b35-073b4480514e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:07:18.385571 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2d088-980f-4f01-9b35-073b4480514e service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:18.386175 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2d088-980f-4f01-9b35-073b4480514e service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:18.386664 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2d088-980f-4f01-9b35-073b4480514e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:18.388498 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-aab2d088-980f-4f01-9b35-073b4480514e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:18.509918 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab2d088-980f-4f01-9b35-073b4480514e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:07:18.511298 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab2d088-980f-4f01-9b35-073b4480514e service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:07:18.532402 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 668/1343] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:07:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:07:21.456641 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adb36b-1128-488b-9983-5520f0d70698 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:21.457526 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adb36b-1128-488b-9983-5520f0d70698 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:21.457526 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adb36b-1128-488b-9983-5520f0d70698 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:21.458632 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 676/1344] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:07:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:07:21.467253 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae200a-4074-451e-9290-0cfb62e7ffdc None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:21.468099 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae200a-4074-451e-9290-0cfb62e7ffdc None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:21.468099 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae200a-4074-451e-9290-0cfb62e7ffdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:21.549355 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-ffae200a-4074-451e-9290-0cfb62e7ffdc None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:07:21.550949 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-ffae200a-4074-451e-9290-0cfb62e7ffdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:21.575799 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:07:21.687733 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 669/1345] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:07:21 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:07:21.727303 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18adb36b-1128-488b-9983-5520f0d70698 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:07:21.729446 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-18adb36b-1128-488b-9983-5520f0d70698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:21.847304 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18adb36b-1128-488b-9983-5520f0d70698 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:07:21.934064 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba1bbe-544b-4759-98ba-6f97bc96864c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:07:21.934712 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba1bbe-544b-4759-98ba-6f97bc96864c service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:21.934989 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba1bbe-544b-4759-98ba-6f97bc96864c service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:21.935255 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba1bbe-544b-4759-98ba-6f97bc96864c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:21.936983 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-75ba1bbe-544b-4759-98ba-6f97bc96864c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:21.986161 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:07:22.063315 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ba1bbe-544b-4759-98ba-6f97bc96864c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:07:22.064918 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ba1bbe-544b-4759-98ba-6f97bc96864c service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:07:22.086560 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 677/1346] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:07:21 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:07:27.115235 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace7206-f8aa-4804-a18b-7d8d7a3ed309 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:27.115858 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace7206-f8aa-4804-a18b-7d8d7a3ed309 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:27.115945 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace7206-f8aa-4804-a18b-7d8d7a3ed309 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:27.117439 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 670/1347] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:07:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:07:27.133667 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d4db2-e852-41d4-8e70-86584bcf9476 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:27.134768 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d4db2-e852-41d4-8e70-86584bcf9476 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:27.134944 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d4db2-e852-41d4-8e70-86584bcf9476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:27.236880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-375d4db2-e852-41d4-8e70-86584bcf9476 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:07:27.238057 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-375d4db2-e852-41d4-8e70-86584bcf9476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:27.385448 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:07:27.391932 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 678/1348] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:07:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:07:27.430591 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace7206-f8aa-4804-a18b-7d8d7a3ed309 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:07:27.434046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-4ace7206-f8aa-4804-a18b-7d8d7a3ed309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:27.584369 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace7206-f8aa-4804-a18b-7d8d7a3ed309 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:07:27.639127 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:07:27.673161 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28160aaa-5b12-48c4-8cac-1b3300ce31bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:07:27.674256 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28160aaa-5b12-48c4-8cac-1b3300ce31bb service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:27.674746 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28160aaa-5b12-48c4-8cac-1b3300ce31bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:27.675211 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28160aaa-5b12-48c4-8cac-1b3300ce31bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:27.677130 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-28160aaa-5b12-48c4-8cac-1b3300ce31bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:27.819246 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28160aaa-5b12-48c4-8cac-1b3300ce31bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:07:27.821416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28160aaa-5b12-48c4-8cac-1b3300ce31bb service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:07:27.848948 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 671/1349] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:07:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:07:32.955040 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fbf241-c71e-42ee-8849-237f0ebc5348 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:32.955602 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fbf241-c71e-42ee-8849-237f0ebc5348 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:32.955602 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fbf241-c71e-42ee-8849-237f0ebc5348 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:32.957464 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 679/1350] 10.222.0.60 () {54 vars in 921 bytes} [Wed Jul 13 23:07:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:07:32.963708 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a069494-30f0-4df2-9665-05a2574c9f8c None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:32.964188 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a069494-30f0-4df2-9665-05a2574c9f8c None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:32.964283 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a069494-30f0-4df2-9665-05a2574c9f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:33.074264 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-5a069494-30f0-4df2-9665-05a2574c9f8c None None] MFA Rules not processed for user `64824ba84a024930bbc177f8a8d9a046`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:07:33.075515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-5a069494-30f0-4df2-9665-05a2574c9f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:33.096475 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:07:33.279390 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 672/1351] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Jul 13 23:07:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:07:37.645016 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c3fb8-752b-49bf-91bb-beb4e1d10482 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:37.645642 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c3fb8-752b-49bf-91bb-beb4e1d10482 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:37.645642 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c3fb8-752b-49bf-91bb-beb4e1d10482 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:37.647283 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 680/1352] 10.222.0.60 () {54 vars in 921 bytes} [Wed Jul 13 23:07:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:07:37.657817 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30306c0b-f7ad-4bf1-adcd-3a28b49b35b1 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:37.658378 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30306c0b-f7ad-4bf1-adcd-3a28b49b35b1 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:37.658378 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30306c0b-f7ad-4bf1-adcd-3a28b49b35b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:37.760317 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-30306c0b-f7ad-4bf1-adcd-3a28b49b35b1 None None] MFA Rules not processed for user `64824ba84a024930bbc177f8a8d9a046`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:07:37.762345 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-30306c0b-f7ad-4bf1-adcd-3a28b49b35b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:37.872043 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:07:37.934468 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 673/1353] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Jul 13 23:07:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:07:39.082575 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941193b-d0dd-49c8-994e-b7e1b4980d4e None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:39.083133 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941193b-d0dd-49c8-994e-b7e1b4980d4e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:39.083133 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941193b-d0dd-49c8-994e-b7e1b4980d4e None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:39.084544 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 681/1354] 10.222.0.60 () {54 vars in 921 bytes} [Wed Jul 13 23:07:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:07:39.090398 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7635b4e-7af5-43f3-8de9-e1d5e6fb7814 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:39.090859 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7635b4e-7af5-43f3-8de9-e1d5e6fb7814 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:39.090978 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7635b4e-7af5-43f3-8de9-e1d5e6fb7814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:39.171865 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-f7635b4e-7af5-43f3-8de9-e1d5e6fb7814 None None] MFA Rules not processed for user `64824ba84a024930bbc177f8a8d9a046`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:07:39.173110 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-f7635b4e-7af5-43f3-8de9-e1d5e6fb7814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:39.292551 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 674/1355] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Jul 13 23:07:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:07:42.888367 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceeb158-cc7d-448c-9781-4c023f3da695 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:42.888932 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceeb158-cc7d-448c-9781-4c023f3da695 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:42.888932 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceeb158-cc7d-448c-9781-4c023f3da695 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:42.890288 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 682/1356] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:07:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:07:42.895670 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72490e6a-ae3f-49e1-a363-6a48c1345973 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:42.896132 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72490e6a-ae3f-49e1-a363-6a48c1345973 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:42.896235 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72490e6a-ae3f-49e1-a363-6a48c1345973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:42.932718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:07:42.966283 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-72490e6a-ae3f-49e1-a363-6a48c1345973 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:07:42.967503 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-72490e6a-ae3f-49e1-a363-6a48c1345973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:43.078968 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 675/1357] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:07:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:07:43.142856 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959151b-68b1-4e5b-8b13-b3156f1042f4 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:43.143232 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959151b-68b1-4e5b-8b13-b3156f1042f4 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:43.143880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959151b-68b1-4e5b-8b13-b3156f1042f4 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:43.150350 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 683/1358] 10.222.0.60 () {54 vars in 961 bytes} [Wed Jul 13 23:07:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:07:43.153726 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037d282-c5c7-4fa6-937f-5f4d671c228f None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:43.154012 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037d282-c5c7-4fa6-937f-5f4d671c228f None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:43.154293 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037d282-c5c7-4fa6-937f-5f4d671c228f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:43.271079 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-3037d282-c5c7-4fa6-937f-5f4d671c228f None None] MFA Rules not processed for user `4789384d158b4995bd30e1e37e30f5f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:07:43.273281 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-3037d282-c5c7-4fa6-937f-5f4d671c228f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:43.336810 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:07:43.439764 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 676/1359] 10.222.0.60 () {58 vars in 1058 bytes} [Wed Jul 13 23:07:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:07:43.442621 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959151b-68b1-4e5b-8b13-b3156f1042f4 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:07:43.444333 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1959151b-68b1-4e5b-8b13-b3156f1042f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:43.618096 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959151b-68b1-4e5b-8b13-b3156f1042f4 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:07:43.712019 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:07:43.767249 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21071b42-75fa-4c68-9b80-367d4b6db39a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:07:43.768687 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21071b42-75fa-4c68-9b80-367d4b6db39a service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:43.769492 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21071b42-75fa-4c68-9b80-367d4b6db39a service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:43.770056 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21071b42-75fa-4c68-9b80-367d4b6db39a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:43.773568 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-21071b42-75fa-4c68-9b80-367d4b6db39a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:43.933313 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21071b42-75fa-4c68-9b80-367d4b6db39a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:07:43.937449 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21071b42-75fa-4c68-9b80-367d4b6db39a service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:07:43.957058 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 684/1360] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:07:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:07:46.287346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259fb9fb-0590-4f29-b748-c129a794968a None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:46.287889 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259fb9fb-0590-4f29-b748-c129a794968a None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:46.287969 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259fb9fb-0590-4f29-b748-c129a794968a None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:46.289589 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 677/1361] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:07:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:07:46.295561 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83e513-5593-4a57-9add-daed937f824c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:46.296055 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83e513-5593-4a57-9add-daed937f824c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:46.296154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83e513-5593-4a57-9add-daed937f824c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:46.370703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-4c83e513-5593-4a57-9add-daed937f824c None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:07:46.371948 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4c83e513-5593-4a57-9add-daed937f824c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:46.411152 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:07:46.513500 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 685/1362] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:07:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:07:46.569688 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259fb9fb-0590-4f29-b748-c129a794968a None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:07:46.572976 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-259fb9fb-0590-4f29-b748-c129a794968a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:46.819358 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259fb9fb-0590-4f29-b748-c129a794968a None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:07:46.836432 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:07:46.974239 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7092c51c-27ab-4184-9b43-9176f13f321c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:07:46.975279 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092c51c-27ab-4184-9b43-9176f13f321c service nova] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:46.975783 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092c51c-27ab-4184-9b43-9176f13f321c service nova] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:46.976248 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092c51c-27ab-4184-9b43-9176f13f321c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:46.979692 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-7092c51c-27ab-4184-9b43-9176f13f321c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:47.095703 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7092c51c-27ab-4184-9b43-9176f13f321c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:07:47.097377 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7092c51c-27ab-4184-9b43-9176f13f321c service nova] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:07:47.121319 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 678/1363] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:07:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:07:49.640340 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469dbee6-de8f-4a69-84bd-5e539ab8af72 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:49.641335 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469dbee6-de8f-4a69-84bd-5e539ab8af72 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:49.641612 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469dbee6-de8f-4a69-84bd-5e539ab8af72 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:49.643058 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 686/1364] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:07:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:07:49.648780 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ef5be6-4b68-4cae-89a7-10c5d1af221c None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:49.649275 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ef5be6-4b68-4cae-89a7-10c5d1af221c None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:49.649370 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ef5be6-4b68-4cae-89a7-10c5d1af221c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:49.667443 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:07:49.731899 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-a4ef5be6-4b68-4cae-89a7-10c5d1af221c None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:07:49.733328 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-a4ef5be6-4b68-4cae-89a7-10c5d1af221c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:49.857054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 679/1365] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:07:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:07:49.903282 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171bd30-2ed2-410d-95ba-ba1df9f90e1c None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:49.903803 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171bd30-2ed2-410d-95ba-ba1df9f90e1c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:49.904164 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171bd30-2ed2-410d-95ba-ba1df9f90e1c None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:49.905924 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 687/1366] 10.222.0.60 () {54 vars in 961 bytes} [Wed Jul 13 23:07:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:07:49.911222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3fcc0-98d1-4e65-a6e8-2867fe325807 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:49.911510 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3fcc0-98d1-4e65-a6e8-2867fe325807 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:49.911830 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3fcc0-98d1-4e65-a6e8-2867fe325807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:49.990641 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-dae3fcc0-98d1-4e65-a6e8-2867fe325807 None None] MFA Rules not processed for user `4789384d158b4995bd30e1e37e30f5f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:07:49.991870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-dae3fcc0-98d1-4e65-a6e8-2867fe325807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:50.028424 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:07:50.120643 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 680/1367] 10.222.0.60 () {58 vars in 1058 bytes} [Wed Jul 13 23:07:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:07:50.128306 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c171bd30-2ed2-410d-95ba-ba1df9f90e1c None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:07:50.129818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c171bd30-2ed2-410d-95ba-ba1df9f90e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:50.278074 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c171bd30-2ed2-410d-95ba-ba1df9f90e1c None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:07:50.299574 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:07:50.413089 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa23ed-e838-4e02-bbb4-e4950f5d2068 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:07:50.413741 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa23ed-e838-4e02-bbb4-e4950f5d2068 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:50.414024 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa23ed-e838-4e02-bbb4-e4950f5d2068 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:50.414308 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa23ed-e838-4e02-bbb4-e4950f5d2068 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:50.415990 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-aaaa23ed-e838-4e02-bbb4-e4950f5d2068 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:50.510325 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaa23ed-e838-4e02-bbb4-e4950f5d2068 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:07:50.511977 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaa23ed-e838-4e02-bbb4-e4950f5d2068 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:07:50.531520 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 688/1368] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:07:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:07:51.503645 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317f031-3cbe-4807-a789-a69cc913b37a None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:51.504170 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317f031-3cbe-4807-a789-a69cc913b37a None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:51.504306 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317f031-3cbe-4807-a789-a69cc913b37a None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:51.505586 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 681/1369] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:07:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:07:51.511806 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c78954-5979-4e22-8cca-ab8b7cddaa72 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:51.512322 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c78954-5979-4e22-8cca-ab8b7cddaa72 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:51.512407 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c78954-5979-4e22-8cca-ab8b7cddaa72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:51.545531 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:07:51.588758 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-79c78954-5979-4e22-8cca-ab8b7cddaa72 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:07:51.589981 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-79c78954-5979-4e22-8cca-ab8b7cddaa72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:51.715401 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 689/1370] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:07:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:07:52.029727 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20575b64-6d7a-41e0-989c-4bc449e2f171 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:52.029727 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20575b64-6d7a-41e0-989c-4bc449e2f171 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:52.029727 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20575b64-6d7a-41e0-989c-4bc449e2f171 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:52.030905 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 682/1371] 10.222.0.60 () {54 vars in 963 bytes} [Wed Jul 13 23:07:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:07:52.038137 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee16f8-e187-497a-8735-293ef1456c60 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:52.038137 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee16f8-e187-497a-8735-293ef1456c60 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:52.038137 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee16f8-e187-497a-8735-293ef1456c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:52.122664 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-91ee16f8-e187-497a-8735-293ef1456c60 None None] MFA Rules not processed for user `54661bad914f455cb00d42ec692fde11`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:07:52.124090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-91ee16f8-e187-497a-8735-293ef1456c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:52.184317 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:07:52.217209 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 690/1372] 10.222.0.60 () {58 vars in 1060 bytes} [Wed Jul 13 23:07:52 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:07:52.225624 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20575b64-6d7a-41e0-989c-4bc449e2f171 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:07:52.227119 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-20575b64-6d7a-41e0-989c-4bc449e2f171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:52.338564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20575b64-6d7a-41e0-989c-4bc449e2f171 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:07:52.393755 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:07:52.432924 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e058a5c-190e-4001-8226-7246e434edfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:07:52.433574 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e058a5c-190e-4001-8226-7246e434edfe service cinder] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:52.434899 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e058a5c-190e-4001-8226-7246e434edfe service cinder] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:52.435250 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e058a5c-190e-4001-8226-7246e434edfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:52.436981 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-7e058a5c-190e-4001-8226-7246e434edfe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:52.569507 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e058a5c-190e-4001-8226-7246e434edfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:07:52.570976 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e058a5c-190e-4001-8226-7246e434edfe service cinder] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:07:52.591277 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 683/1373] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:07:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:07:52.650969 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b4ce08-a4c5-4e16-993d-16e1489f9d5e None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:52.651398 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b4ce08-a4c5-4e16-993d-16e1489f9d5e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:52.651732 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b4ce08-a4c5-4e16-993d-16e1489f9d5e None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:52.653312 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 691/1374] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:07:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:07:52.658692 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d735b9-3d54-4baf-9bcb-cd8a5141581d None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:52.658993 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d735b9-3d54-4baf-9bcb-cd8a5141581d None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:52.659252 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d735b9-3d54-4baf-9bcb-cd8a5141581d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:52.733119 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-e5d735b9-3d54-4baf-9bcb-cd8a5141581d None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:07:52.734326 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-e5d735b9-3d54-4baf-9bcb-cd8a5141581d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:52.759161 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:07:52.860217 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 684/1375] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:07:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:07:52.928528 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b4ce08-a4c5-4e16-993d-16e1489f9d5e None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:07:52.931282 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-73b4ce08-a4c5-4e16-993d-16e1489f9d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:53.138444 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b4ce08-a4c5-4e16-993d-16e1489f9d5e None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:07:53.190417 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:07:53.254960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989cf9f9-3a5f-416d-982b-f47b2099588c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:07:53.255641 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cf9f9-3a5f-416d-982b-f47b2099588c service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:53.255931 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cf9f9-3a5f-416d-982b-f47b2099588c service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:53.256209 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cf9f9-3a5f-416d-982b-f47b2099588c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:53.257882 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-989cf9f9-3a5f-416d-982b-f47b2099588c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:53.370437 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989cf9f9-3a5f-416d-982b-f47b2099588c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:07:53.370549 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989cf9f9-3a5f-416d-982b-f47b2099588c service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:07:53.390094 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 692/1376] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:07:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:07:55.450303 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8019a0-23c2-41c1-a3d4-39f00e178176 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:55.450855 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8019a0-23c2-41c1-a3d4-39f00e178176 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:55.450855 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8019a0-23c2-41c1-a3d4-39f00e178176 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:55.452349 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 685/1377] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:07:55.457783 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159fd92-96a6-4112-a34b-43f54a48a14a None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:55.458230 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159fd92-96a6-4112-a34b-43f54a48a14a None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:55.458454 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159fd92-96a6-4112-a34b-43f54a48a14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:55.526809 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-6159fd92-96a6-4112-a34b-43f54a48a14a None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:07:55.528078 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6159fd92-96a6-4112-a34b-43f54a48a14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:55.535245 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:07:55.637416 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 693/1378] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:07:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:07:55.683349 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8019a0-23c2-41c1-a3d4-39f00e178176 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:07:55.684995 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-cf8019a0-23c2-41c1-a3d4-39f00e178176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:55.861705 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8019a0-23c2-41c1-a3d4-39f00e178176 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:07:55.865829 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:07:55.993564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c02ef55-9334-4e68-86fa-559a5aa89ed4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:07:55.994217 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c02ef55-9334-4e68-86fa-559a5aa89ed4 service nova] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:55.994495 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c02ef55-9334-4e68-86fa-559a5aa89ed4 service nova] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:55.994768 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c02ef55-9334-4e68-86fa-559a5aa89ed4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:55.996443 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-0c02ef55-9334-4e68-86fa-559a5aa89ed4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:56.104545 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c02ef55-9334-4e68-86fa-559a5aa89ed4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:07:56.107095 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c02ef55-9334-4e68-86fa-559a5aa89ed4 service nova] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:07:56.134613 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 686/1379] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:07:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:07:58.328780 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f215a-f36b-409a-a4f3-ea6e90f50432 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:58.329953 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f215a-f36b-409a-a4f3-ea6e90f50432 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:58.330484 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f215a-f36b-409a-a4f3-ea6e90f50432 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:58.333209 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 694/1380] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:07:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:07:58.338424 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5a759-b481-481f-95fa-5ddd7933de01 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:58.338880 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5a759-b481-481f-95fa-5ddd7933de01 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:58.338966 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5a759-b481-481f-95fa-5ddd7933de01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:58.349759 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:07:58.427147 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-3bb5a759-b481-481f-95fa-5ddd7933de01 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:07:58.428778 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-3bb5a759-b481-481f-95fa-5ddd7933de01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:58.543772 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 687/1381] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:07:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:07:58.588225 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f215a-f36b-409a-a4f3-ea6e90f50432 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:07:58.589762 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-470f215a-f36b-409a-a4f3-ea6e90f50432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:58.783398 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:07:58.801204 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f215a-f36b-409a-a4f3-ea6e90f50432 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:07:58.937820 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f7e77-4de2-421e-baec-c35697b8b036 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:07:58.938720 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f7e77-4de2-421e-baec-c35697b8b036 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:07:58.938999 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f7e77-4de2-421e-baec-c35697b8b036 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:07:58.939267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f7e77-4de2-421e-baec-c35697b8b036 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:07:58.940952 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a29f7e77-4de2-421e-baec-c35697b8b036 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:07:59.057727 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f7e77-4de2-421e-baec-c35697b8b036 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:07:59.059883 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f7e77-4de2-421e-baec-c35697b8b036 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:07:59.074606 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:07:59.088056 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 695/1382] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:07:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:08:01.470430 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40695898-3a1e-41c9-9cf0-56df300d2fb9 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:01.470970 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40695898-3a1e-41c9-9cf0-56df300d2fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:01.471384 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40695898-3a1e-41c9-9cf0-56df300d2fb9 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:01.473724 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 688/1383] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:08:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:08:01.479320 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af12187-4e3a-47de-9987-58747af61801 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:01.479808 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af12187-4e3a-47de-9987-58747af61801 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:01.479928 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af12187-4e3a-47de-9987-58747af61801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:01.547453 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-1af12187-4e3a-47de-9987-58747af61801 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:08:01.548715 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1af12187-4e3a-47de-9987-58747af61801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:01.664440 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 696/1384] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:08:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:08:01.707239 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40695898-3a1e-41c9-9cf0-56df300d2fb9 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:08:01.709873 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-40695898-3a1e-41c9-9cf0-56df300d2fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:01.848952 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:01.926468 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40695898-3a1e-41c9-9cf0-56df300d2fb9 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:08:02.041686 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304945c7-dd93-4338-a458-47cb82868a1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:08:02.042336 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304945c7-dd93-4338-a458-47cb82868a1d service nova] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:02.042628 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304945c7-dd93-4338-a458-47cb82868a1d service nova] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:02.042872 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304945c7-dd93-4338-a458-47cb82868a1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:02.044581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-304945c7-dd93-4338-a458-47cb82868a1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:02.095053 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:02.136868 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304945c7-dd93-4338-a458-47cb82868a1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:08:02.138508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304945c7-dd93-4338-a458-47cb82868a1d service nova] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:08:02.162109 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 689/1385] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:08:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:08:04.623632 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f2204-3c88-436c-8f3d-9885fc779ba3 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:04.624225 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f2204-3c88-436c-8f3d-9885fc779ba3 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:04.624225 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f2204-3c88-436c-8f3d-9885fc779ba3 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:04.625570 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 697/1386] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:08:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:08:04.632005 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6416a67-ee70-491e-81c3-b92721d970d6 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:04.632471 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6416a67-ee70-491e-81c3-b92721d970d6 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:04.632553 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6416a67-ee70-491e-81c3-b92721d970d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:04.709915 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-f6416a67-ee70-491e-81c3-b92721d970d6 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:08:04.711128 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-f6416a67-ee70-491e-81c3-b92721d970d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:04.817796 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 690/1387] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:08:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:08:04.862896 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083f2204-3c88-436c-8f3d-9885fc779ba3 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:08:04.864459 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-083f2204-3c88-436c-8f3d-9885fc779ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:04.939190 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:05.018938 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083f2204-3c88-436c-8f3d-9885fc779ba3 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:08:05.138519 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558cc9d6-d875-4ae3-a1e5-8e75b9701b87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:08:05.139196 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558cc9d6-d875-4ae3-a1e5-8e75b9701b87 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:05.139475 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558cc9d6-d875-4ae3-a1e5-8e75b9701b87 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:05.139777 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558cc9d6-d875-4ae3-a1e5-8e75b9701b87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:05.141421 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-558cc9d6-d875-4ae3-a1e5-8e75b9701b87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:05.180205 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:05.233021 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558cc9d6-d875-4ae3-a1e5-8e75b9701b87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:08:05.234683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558cc9d6-d875-4ae3-a1e5-8e75b9701b87 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:08:05.253548 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 698/1388] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:08:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:08:07.457211 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56226120-ea45-498d-9b57-2576ae160797 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:07.457806 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56226120-ea45-498d-9b57-2576ae160797 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:07.457806 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56226120-ea45-498d-9b57-2576ae160797 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:07.459131 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 691/1389] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:08:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:08:07.464821 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5032827-a2b1-4f89-97b8-87eea9cf7aac None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:07.465297 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5032827-a2b1-4f89-97b8-87eea9cf7aac None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:07.465403 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5032827-a2b1-4f89-97b8-87eea9cf7aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:07.544984 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c5032827-a2b1-4f89-97b8-87eea9cf7aac None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:08:07.546256 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c5032827-a2b1-4f89-97b8-87eea9cf7aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:07.687594 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 699/1390] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:08:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:08:07.730903 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56226120-ea45-498d-9b57-2576ae160797 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:08:07.732548 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-56226120-ea45-498d-9b57-2576ae160797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:07.764181 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:07.879739 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56226120-ea45-498d-9b57-2576ae160797 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:08:07.987410 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:08.011884 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee4945a-a658-4d64-8540-a8933936715b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:08:08.012565 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4945a-a658-4d64-8540-a8933936715b service nova] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:08.012849 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4945a-a658-4d64-8540-a8933936715b service nova] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:08.013121 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee4945a-a658-4d64-8540-a8933936715b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:08.014918 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-3ee4945a-a658-4d64-8540-a8933936715b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:08.138667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee4945a-a658-4d64-8540-a8933936715b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:08:08.140391 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee4945a-a658-4d64-8540-a8933936715b service nova] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:08:08.158988 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 692/1391] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:08:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:08:08.172319 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03564f6b-1ce1-42cf-ae38-6d5a7d222da1 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:08.172824 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03564f6b-1ce1-42cf-ae38-6d5a7d222da1 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:08.173101 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03564f6b-1ce1-42cf-ae38-6d5a7d222da1 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:08.174503 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 700/1392] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:08:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:08:08.181803 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c874c-a8c9-4ff1-a575-f261f759cfac None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:08.182087 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c874c-a8c9-4ff1-a575-f261f759cfac None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:08.182359 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c874c-a8c9-4ff1-a575-f261f759cfac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:08.255087 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-891c874c-a8c9-4ff1-a575-f261f759cfac None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:08:08.256296 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-891c874c-a8c9-4ff1-a575-f261f759cfac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:08.325238 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:08.370405 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 693/1393] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:08:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:08:08.426173 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f49bc-d6b4-4ec8-909c-03c5d9e3229f None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:08.426450 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f49bc-d6b4-4ec8-909c-03c5d9e3229f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:08.426722 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f49bc-d6b4-4ec8-909c-03c5d9e3229f None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:08.428125 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 701/1394] 10.222.0.60 () {54 vars in 962 bytes} [Wed Jul 13 23:08:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:08:08.433389 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7d930-13fa-4af8-a536-c402af4952c4 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:08.433679 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7d930-13fa-4af8-a536-c402af4952c4 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:08.434048 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7d930-13fa-4af8-a536-c402af4952c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:08.511574 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-d7b7d930-13fa-4af8-a536-c402af4952c4 None None] MFA Rules not processed for user `2ce9e6497fdb4068b23aa3f7b2c19536`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:08:08.512736 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-d7b7d930-13fa-4af8-a536-c402af4952c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:08.595043 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 694/1395] 10.222.0.60 () {58 vars in 1059 bytes} [Wed Jul 13 23:08:08 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:08:08.603309 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f49bc-d6b4-4ec8-909c-03c5d9e3229f None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:08:08.604861 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-847f49bc-d6b4-4ec8-909c-03c5d9e3229f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:08.618498 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:08.714103 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f49bc-d6b4-4ec8-909c-03c5d9e3229f None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:08:08.789742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab62d4-eee7-403a-a8f0-cb9ebb683c48 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ce9e6497fdb4068b23aa3f7b2c19536', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:08:08.790350 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab62d4-eee7-403a-a8f0-cb9ebb683c48 service glance] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:08.790634 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab62d4-eee7-403a-a8f0-cb9ebb683c48 service glance] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:08.790906 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab62d4-eee7-403a-a8f0-cb9ebb683c48 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:08.792715 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0eab62d4-eee7-403a-a8f0-cb9ebb683c48 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:08.866775 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:08.920183 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eab62d4-eee7-403a-a8f0-cb9ebb683c48 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:08:08.921791 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eab62d4-eee7-403a-a8f0-cb9ebb683c48 service glance] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:08:08.943385 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 702/1396] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:08:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:08:09.660087 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a01f35-275a-4d5a-9312-884586e44cf9 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:09.660595 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a01f35-275a-4d5a-9312-884586e44cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:09.660804 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a01f35-275a-4d5a-9312-884586e44cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:09.727810 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-16a01f35-275a-4d5a-9312-884586e44cf9 None None] MFA Rules not processed for user `a32702a67a7b49ffa153fc59251fd169`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:08:09.729021 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-16a01f35-275a-4d5a-9312-884586e44cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:09.737576 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:09.819480 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 695/1397] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Jul 13 23:08:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:08:09.869177 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37966a90-93c8-4b00-a659-ec3a332760a9 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:09.869709 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37966a90-93c8-4b00-a659-ec3a332760a9 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:09.869835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37966a90-93c8-4b00-a659-ec3a332760a9 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:09.871305 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 703/1398] 10.222.0.60 () {54 vars in 942 bytes} [Wed Jul 13 23:08:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:08:09.878044 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b17cd-97a2-4444-96dd-fa831215b4d5 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:09.878336 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b17cd-97a2-4444-96dd-fa831215b4d5 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:09.878622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b17cd-97a2-4444-96dd-fa831215b4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:09.957407 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-277b17cd-97a2-4444-96dd-fa831215b4d5 None None] MFA Rules not processed for user `786fc44611804d3ca6280ffcab415569`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:08:09.958566 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-277b17cd-97a2-4444-96dd-fa831215b4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:10.046592 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:10.076768 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 696/1399] 10.222.0.60 () {58 vars in 1039 bytes} [Wed Jul 13 23:08:09 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:08:10.086776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37966a90-93c8-4b00-a659-ec3a332760a9 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:08:10.088449 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-37966a90-93c8-4b00-a659-ec3a332760a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:10.230737 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37966a90-93c8-4b00-a659-ec3a332760a9 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:08:10.253636 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af28924-d7a3-41a7-a682-a9c5296ebac3 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:10.253923 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af28924-d7a3-41a7-a682-a9c5296ebac3 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:10.254201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af28924-d7a3-41a7-a682-a9c5296ebac3 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:10.255610 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 697/1400] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:08:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:08:10.261116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da94b3ae-a8e2-4c2c-a0a3-4dbfa071143f None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:10.261747 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da94b3ae-a8e2-4c2c-a0a3-4dbfa071143f None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:10.261747 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da94b3ae-a8e2-4c2c-a0a3-4dbfa071143f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:10.271833 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:10.338053 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-da94b3ae-a8e2-4c2c-a0a3-4dbfa071143f None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:08:10.339311 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-da94b3ae-a8e2-4c2c-a0a3-4dbfa071143f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:10.343344 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb176d8-5521-44f6-87d2-22736a516ba7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786fc44611804d3ca6280ffcab415569', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:08:10.344024 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb176d8-5521-44f6-87d2-22736a516ba7 service swift] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:10.344305 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb176d8-5521-44f6-87d2-22736a516ba7 service swift] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:10.344577 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb176d8-5521-44f6-87d2-22736a516ba7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:10.346271 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ebb176d8-5521-44f6-87d2-22736a516ba7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:10.440347 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb176d8-5521-44f6-87d2-22736a516ba7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:08:10.441956 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb176d8-5521-44f6-87d2-22736a516ba7 service swift] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:08:10.446543 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 698/1401] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:08:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:08:10.450219 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 704/1402] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Jul 13 23:08:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 23:08:10.491937 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da94b3ae-a8e2-4c2c-a0a3-4dbfa071143f None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:08:10.493284 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-da94b3ae-a8e2-4c2c-a0a3-4dbfa071143f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:10.589237 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:10.667135 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da94b3ae-a8e2-4c2c-a0a3-4dbfa071143f None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:08:10.830149 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f3b94-4f78-4d2b-aba8-db2dcc9152f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:08:10.831086 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f3b94-4f78-4d2b-aba8-db2dcc9152f5 service nova] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:10.831373 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f3b94-4f78-4d2b-aba8-db2dcc9152f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:10.831692 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f3b94-4f78-4d2b-aba8-db2dcc9152f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:10.833429 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-661f3b94-4f78-4d2b-aba8-db2dcc9152f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:10.910696 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:10.960075 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f3b94-4f78-4d2b-aba8-db2dcc9152f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:08:10.963306 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f3b94-4f78-4d2b-aba8-db2dcc9152f5 service nova] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:08:11.000667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 699/1403] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:08:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:08:13.112445 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f0d71-8985-465b-90b6-69e1c94d858f None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:13.113339 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f0d71-8985-465b-90b6-69e1c94d858f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:13.113339 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f0d71-8985-465b-90b6-69e1c94d858f None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:13.114789 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 705/1404] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:08:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:08:13.121677 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd960b25-14e0-440b-b8a6-47bb0bc425e2 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:13.122199 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd960b25-14e0-440b-b8a6-47bb0bc425e2 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:13.122482 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd960b25-14e0-440b-b8a6-47bb0bc425e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:13.220027 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-dd960b25-14e0-440b-b8a6-47bb0bc425e2 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:08:13.222004 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-dd960b25-14e0-440b-b8a6-47bb0bc425e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:13.342190 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 700/1405] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:08:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:08:13.388341 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f0d71-8985-465b-90b6-69e1c94d858f None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:08:13.389872 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-268f0d71-8985-465b-90b6-69e1c94d858f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:13.478565 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:13.600609 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f0d71-8985-465b-90b6-69e1c94d858f None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:08:13.766841 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e804d6f-a760-46d8-9fb0-7b80209c48a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:08:13.767915 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e804d6f-a760-46d8-9fb0-7b80209c48a2 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:13.768198 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e804d6f-a760-46d8-9fb0-7b80209c48a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:13.768475 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e804d6f-a760-46d8-9fb0-7b80209c48a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:13.770196 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9e804d6f-a760-46d8-9fb0-7b80209c48a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:13.810001 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:13.881059 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e804d6f-a760-46d8-9fb0-7b80209c48a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:08:13.882765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e804d6f-a760-46d8-9fb0-7b80209c48a2 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:08:13.906109 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 706/1406] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:08:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:08:16.048512 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a8766-33a9-46c1-9739-34a590935958 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:16.048512 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a8766-33a9-46c1-9739-34a590935958 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:16.048512 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a8766-33a9-46c1-9739-34a590935958 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:16.048512 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 701/1407] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:08:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:08:16.049356 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb3127-c3ab-4f83-85a5-f23cfd79ce4a None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:16.049356 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb3127-c3ab-4f83-85a5-f23cfd79ce4a None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:16.049356 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb3127-c3ab-4f83-85a5-f23cfd79ce4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:16.205402 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-5acb3127-c3ab-4f83-85a5-f23cfd79ce4a None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:08:16.206664 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5acb3127-c3ab-4f83-85a5-f23cfd79ce4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:16.372048 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 707/1408] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:08:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:08:16.420560 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a8766-33a9-46c1-9739-34a590935958 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:08:16.422090 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-092a8766-33a9-46c1-9739-34a590935958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:16.475044 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:16.708537 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a8766-33a9-46c1-9739-34a590935958 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:08:16.902261 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:16.911324 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ae4dc-d7f6-46c8-9d05-da05d637b71c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:08:16.912136 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ae4dc-d7f6-46c8-9d05-da05d637b71c service nova] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:16.912489 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ae4dc-d7f6-46c8-9d05-da05d637b71c service nova] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:16.912829 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ae4dc-d7f6-46c8-9d05-da05d637b71c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:16.914681 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-9e7ae4dc-d7f6-46c8-9d05-da05d637b71c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:17.035239 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7ae4dc-d7f6-46c8-9d05-da05d637b71c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:08:17.037035 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7ae4dc-d7f6-46c8-9d05-da05d637b71c service nova] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:08:17.061875 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 702/1409] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:08:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 641 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:08:26.369340 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:27.056731 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=47 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:32.725773 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07c558-e220-424e-b413-61cf0a04ecfc None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:32.726346 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07c558-e220-424e-b413-61cf0a04ecfc None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:32.726346 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07c558-e220-424e-b413-61cf0a04ecfc None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:32.727843 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 708/1410] 10.222.0.60 () {54 vars in 918 bytes} [Wed Jul 13 23:08:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:08:32.732990 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57279f76-e075-449c-9615-38976b903337 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:32.733435 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57279f76-e075-449c-9615-38976b903337 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:32.733542 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57279f76-e075-449c-9615-38976b903337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:32.809804 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-57279f76-e075-449c-9615-38976b903337 None None] MFA Rules not processed for user `64824ba84a024930bbc177f8a8d9a046`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:08:32.811080 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-57279f76-e075-449c-9615-38976b903337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:32.916554 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 703/1411] 10.222.0.60 () {58 vars in 1015 bytes} [Wed Jul 13 23:08:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:08:32.931489 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa9a8f-f3e8-45cb-ad96-ffdb48c38047 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:32.931808 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa9a8f-f3e8-45cb-ad96-ffdb48c38047 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:32.932089 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa9a8f-f3e8-45cb-ad96-ffdb48c38047 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:32.933516 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 709/1412] 10.222.0.60 () {54 vars in 918 bytes} [Wed Jul 13 23:08:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:08:32.938182 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3040c88f-fa9b-499c-aca0-958953b6bd26 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:32.938473 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3040c88f-fa9b-499c-aca0-958953b6bd26 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:32.939004 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3040c88f-fa9b-499c-aca0-958953b6bd26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:33.011579 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:33.015189 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-3040c88f-fa9b-499c-aca0-958953b6bd26 None None] MFA Rules not processed for user `4789384d158b4995bd30e1e37e30f5f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:08:33.016755 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-3040c88f-fa9b-499c-aca0-958953b6bd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:33.132668 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 704/1413] 10.222.0.60 () {58 vars in 1015 bytes} [Wed Jul 13 23:08:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:08:35.708098 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1380f-470e-44d2-87c0-737eb924f9a6 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:35.708704 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1380f-470e-44d2-87c0-737eb924f9a6 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:35.708805 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1380f-470e-44d2-87c0-737eb924f9a6 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:35.710135 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 710/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 23:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:08:35.715627 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c29a0-6e18-4966-884f-55336fd30c56 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:35.716100 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c29a0-6e18-4966-884f-55336fd30c56 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:35.716218 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75c29a0-6e18-4966-884f-55336fd30c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:35.783695 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-f75c29a0-6e18-4966-884f-55336fd30c56 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:08:35.784918 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-f75c29a0-6e18-4966-884f-55336fd30c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:35.841014 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:35.885049 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 705/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 23:08:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:08:35.927814 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1380f-470e-44d2-87c0-737eb924f9a6 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:08:35.929640 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0ba1380f-470e-44d2-87c0-737eb924f9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:36.079191 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1380f-470e-44d2-87c0-737eb924f9a6 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:08:36.153879 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:36.197000 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d30994f-93c6-4725-8677-71e2e8a7ccf1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:08:36.197651 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30994f-93c6-4725-8677-71e2e8a7ccf1 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:36.197934 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30994f-93c6-4725-8677-71e2e8a7ccf1 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:36.198210 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d30994f-93c6-4725-8677-71e2e8a7ccf1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:36.199937 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4d30994f-93c6-4725-8677-71e2e8a7ccf1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:36.302443 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d30994f-93c6-4725-8677-71e2e8a7ccf1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:08:36.304153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d30994f-93c6-4725-8677-71e2e8a7ccf1 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:08:36.325515 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 711/1416] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:08:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:08:37.256014 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7a382-9f08-4fbe-9214-1d3a964c52e5 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:37.256614 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7a382-9f08-4fbe-9214-1d3a964c52e5 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:37.256614 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7a382-9f08-4fbe-9214-1d3a964c52e5 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:37.257939 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 706/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 23:08:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:08:37.263695 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea73dec6-9cc6-468d-bb2d-11b7f3ced49a None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:37.264167 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea73dec6-9cc6-468d-bb2d-11b7f3ced49a None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:37.264264 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea73dec6-9cc6-468d-bb2d-11b7f3ced49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:37.340531 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-ea73dec6-9cc6-468d-bb2d-11b7f3ced49a None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:08:37.341763 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ea73dec6-9cc6-468d-bb2d-11b7f3ced49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:37.370704 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:37.443078 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 712/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 23:08:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:08:37.594583 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be7a382-9f08-4fbe-9214-1d3a964c52e5 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:08:37.596074 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-9be7a382-9f08-4fbe-9214-1d3a964c52e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:37.745488 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be7a382-9f08-4fbe-9214-1d3a964c52e5 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:08:37.785493 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:37.863067 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb30bab1-8572-4e5d-ab12-e09b6de5e895 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:08:37.863681 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30bab1-8572-4e5d-ab12-e09b6de5e895 service nova] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:37.863964 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30bab1-8572-4e5d-ab12-e09b6de5e895 service nova] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:37.864238 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30bab1-8572-4e5d-ab12-e09b6de5e895 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:37.866010 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-bb30bab1-8572-4e5d-ab12-e09b6de5e895 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:37.958566 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb30bab1-8572-4e5d-ab12-e09b6de5e895 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:08:37.960304 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb30bab1-8572-4e5d-ab12-e09b6de5e895 service nova] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:08:37.980828 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 707/1419] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:08:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:08:38.876764 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ffa80a-65aa-409f-a7ad-b65089c84b82 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:38.877337 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ffa80a-65aa-409f-a7ad-b65089c84b82 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:38.877337 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ffa80a-65aa-409f-a7ad-b65089c84b82 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:38.878751 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 713/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 23:08:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:08:38.884784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558cbcd-eb93-46a2-a02b-8fb328236304 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:38.885264 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558cbcd-eb93-46a2-a02b-8fb328236304 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:38.885363 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558cbcd-eb93-46a2-a02b-8fb328236304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:38.942193 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:38.965674 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-1558cbcd-eb93-46a2-a02b-8fb328236304 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:08:38.966903 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-1558cbcd-eb93-46a2-a02b-8fb328236304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:39.078595 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 708/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 23:08:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:08:39.231880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ffa80a-65aa-409f-a7ad-b65089c84b82 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:08:39.233831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-26ffa80a-65aa-409f-a7ad-b65089c84b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:39.389869 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ffa80a-65aa-409f-a7ad-b65089c84b82 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:08:39.406051 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:39.518396 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5040b811-9adc-42e0-a4f6-a0e0d1f6d28e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:08:39.518966 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040b811-9adc-42e0-a4f6-a0e0d1f6d28e service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:39.519253 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040b811-9adc-42e0-a4f6-a0e0d1f6d28e service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:39.519566 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040b811-9adc-42e0-a4f6-a0e0d1f6d28e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:39.521317 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5040b811-9adc-42e0-a4f6-a0e0d1f6d28e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:39.628694 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5040b811-9adc-42e0-a4f6-a0e0d1f6d28e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:08:39.630393 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5040b811-9adc-42e0-a4f6-a0e0d1f6d28e service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:08:39.650627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 714/1422] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:08:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:08:40.521629 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e0535e-c6ad-4dd5-b5be-aff3f25ad458 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:40.522614 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e0535e-c6ad-4dd5-b5be-aff3f25ad458 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:40.522614 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e0535e-c6ad-4dd5-b5be-aff3f25ad458 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:40.524346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 709/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 23:08:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:08:40.530246 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f34a0a-040e-4869-92d1-52e234cd29ba None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:40.531075 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f34a0a-040e-4869-92d1-52e234cd29ba None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:40.531287 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f34a0a-040e-4869-92d1-52e234cd29ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:40.552339 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:40.617306 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-f0f34a0a-040e-4869-92d1-52e234cd29ba None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:08:40.619099 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f0f34a0a-040e-4869-92d1-52e234cd29ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:40.723033 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 715/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 23:08:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:08:40.874880 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e0535e-c6ad-4dd5-b5be-aff3f25ad458 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:08:40.876358 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-58e0535e-c6ad-4dd5-b5be-aff3f25ad458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:40.987081 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:41.022747 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e0535e-c6ad-4dd5-b5be-aff3f25ad458 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:08:41.136638 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9c391-b02e-4ded-bade-cd92abfd31a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:08:41.137237 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9c391-b02e-4ded-bade-cd92abfd31a4 service nova] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:41.137522 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9c391-b02e-4ded-bade-cd92abfd31a4 service nova] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:41.137799 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9c391-b02e-4ded-bade-cd92abfd31a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:41.139487 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-46b9c391-b02e-4ded-bade-cd92abfd31a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:41.216267 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:41.230514 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b9c391-b02e-4ded-bade-cd92abfd31a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:08:41.232198 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b9c391-b02e-4ded-bade-cd92abfd31a4 service nova] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:08:41.250398 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 710/1425] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:08:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:08:42.126182 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53564ca-388a-4c3d-9aba-1a03f309de56 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:42.126913 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53564ca-388a-4c3d-9aba-1a03f309de56 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:42.126913 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53564ca-388a-4c3d-9aba-1a03f309de56 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:42.128326 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 716/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 23:08:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:08:42.134002 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c881e18-c53c-459e-a7a3-87703d164bfa None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:42.134555 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c881e18-c53c-459e-a7a3-87703d164bfa None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:42.134555 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c881e18-c53c-459e-a7a3-87703d164bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:42.211300 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-9c881e18-c53c-459e-a7a3-87703d164bfa None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:08:42.212645 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-9c881e18-c53c-459e-a7a3-87703d164bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:42.315050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 711/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 23:08:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:08:42.466741 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53564ca-388a-4c3d-9aba-1a03f309de56 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:08:42.468836 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c53564ca-388a-4c3d-9aba-1a03f309de56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:42.556787 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:42.620223 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53564ca-388a-4c3d-9aba-1a03f309de56 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:08:42.744387 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb2a46-7cc2-47d1-aeb2-c79148015a3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:08:42.744932 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb2a46-7cc2-47d1-aeb2-c79148015a3f service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:42.745218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb2a46-7cc2-47d1-aeb2-c79148015a3f service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:42.745492 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb2a46-7cc2-47d1-aeb2-c79148015a3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:42.747188 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1ffb2a46-7cc2-47d1-aeb2-c79148015a3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:42.803948 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:42.848691 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffb2a46-7cc2-47d1-aeb2-c79148015a3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:08:42.850347 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffb2a46-7cc2-47d1-aeb2-c79148015a3f service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:08:42.869591 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 717/1428] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:08:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:08:43.675572 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f44ca8-7281-42fe-9335-0d84b5c9be58 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:43.676231 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f44ca8-7281-42fe-9335-0d84b5c9be58 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:43.676231 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f44ca8-7281-42fe-9335-0d84b5c9be58 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:43.677911 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 712/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 23:08:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:08:43.685865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044d3af-ef4a-4afb-86dc-88feeb4f52a7 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:43.686475 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044d3af-ef4a-4afb-86dc-88feeb4f52a7 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:43.686475 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044d3af-ef4a-4afb-86dc-88feeb4f52a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:43.765449 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-2044d3af-ef4a-4afb-86dc-88feeb4f52a7 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:08:43.766708 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-2044d3af-ef4a-4afb-86dc-88feeb4f52a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:43.874900 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 718/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 23:08:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:08:44.023886 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f44ca8-7281-42fe-9335-0d84b5c9be58 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:08:44.025964 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-a2f44ca8-7281-42fe-9335-0d84b5c9be58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:44.077823 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:44.178297 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f44ca8-7281-42fe-9335-0d84b5c9be58 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:08:44.290717 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5545193f-becc-40b5-b4d5-67d9c3831eff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:08:44.291273 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545193f-becc-40b5-b4d5-67d9c3831eff service nova] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:44.291572 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545193f-becc-40b5-b4d5-67d9c3831eff service nova] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:44.291846 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545193f-becc-40b5-b4d5-67d9c3831eff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:44.293520 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-5545193f-becc-40b5-b4d5-67d9c3831eff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:44.300489 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:44.384022 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5545193f-becc-40b5-b4d5-67d9c3831eff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:08:44.385662 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5545193f-becc-40b5-b4d5-67d9c3831eff service nova] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:08:44.405549 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 713/1431] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:08:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:08:45.223710 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f08d01-df83-4eb5-bcf3-00c806f02054 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:45.224300 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f08d01-df83-4eb5-bcf3-00c806f02054 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:45.224300 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f08d01-df83-4eb5-bcf3-00c806f02054 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:45.225651 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 719/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 23:08:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:08:45.231144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc46a93-626c-4361-9f56-644fb7329faf None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:45.231631 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc46a93-626c-4361-9f56-644fb7329faf None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:45.231762 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc46a93-626c-4361-9f56-644fb7329faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:45.307788 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-cfc46a93-626c-4361-9f56-644fb7329faf None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:08:45.308986 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-cfc46a93-626c-4361-9f56-644fb7329faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:45.378646 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:45.407712 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 714/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 23:08:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:08:45.557223 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f08d01-df83-4eb5-bcf3-00c806f02054 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:08:45.558674 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d0f08d01-df83-4eb5-bcf3-00c806f02054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:45.576346 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:45.705935 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f08d01-df83-4eb5-bcf3-00c806f02054 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:08:45.792275 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:45.818423 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc1de2b-b80f-4baf-bd65-80b258ca0246 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:08:45.819067 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc1de2b-b80f-4baf-bd65-80b258ca0246 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:45.819349 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc1de2b-b80f-4baf-bd65-80b258ca0246 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:45.819651 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc1de2b-b80f-4baf-bd65-80b258ca0246 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:45.821832 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5fc1de2b-b80f-4baf-bd65-80b258ca0246 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:45.909496 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc1de2b-b80f-4baf-bd65-80b258ca0246 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:08:45.911101 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc1de2b-b80f-4baf-bd65-80b258ca0246 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:08:45.928892 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 720/1434] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:08:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:08:46.740994 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704b47aa-d2cb-4a77-af8b-8e4d95279537 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:46.741553 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704b47aa-d2cb-4a77-af8b-8e4d95279537 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:46.741553 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704b47aa-d2cb-4a77-af8b-8e4d95279537 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:46.742996 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 715/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 23:08:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:08:46.748011 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850c611-9531-4607-aebf-562d9fdb5a5c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:46.748451 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850c611-9531-4607-aebf-562d9fdb5a5c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:46.748561 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850c611-9531-4607-aebf-562d9fdb5a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:46.823112 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-2850c611-9531-4607-aebf-562d9fdb5a5c None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:08:46.824375 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-2850c611-9531-4607-aebf-562d9fdb5a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:46.866269 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:46.921817 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 721/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 23:08:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:08:47.074593 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704b47aa-d2cb-4a77-af8b-8e4d95279537 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:08:47.076132 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-704b47aa-d2cb-4a77-af8b-8e4d95279537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:47.228558 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704b47aa-d2cb-4a77-af8b-8e4d95279537 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:08:47.284234 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:47.344854 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b216a-6718-4e6d-be85-73364d5cca45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:08:47.345511 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b216a-6718-4e6d-be85-73364d5cca45 service nova] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:47.346095 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b216a-6718-4e6d-be85-73364d5cca45 service nova] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:47.346095 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b216a-6718-4e6d-be85-73364d5cca45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:47.347760 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-711b216a-6718-4e6d-be85-73364d5cca45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:47.439394 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b216a-6718-4e6d-be85-73364d5cca45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:08:47.441067 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b216a-6718-4e6d-be85-73364d5cca45 service nova] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:08:47.459139 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 716/1437] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:08:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:08:48.289377 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366a659-1bdd-4ba1-93d0-d7c5b24535ea None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:48.289992 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366a659-1bdd-4ba1-93d0-d7c5b24535ea None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:48.289992 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366a659-1bdd-4ba1-93d0-d7c5b24535ea None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:48.291297 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 722/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 23:08:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:08:48.296396 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4254ee39-db0e-4389-86cb-0a9cfaec66fa None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:48.296804 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4254ee39-db0e-4389-86cb-0a9cfaec66fa None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:48.296961 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4254ee39-db0e-4389-86cb-0a9cfaec66fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:48.373003 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-4254ee39-db0e-4389-86cb-0a9cfaec66fa None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:08:48.374218 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-4254ee39-db0e-4389-86cb-0a9cfaec66fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:48.376311 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:48.472701 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 717/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 23:08:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:08:48.622764 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8366a659-1bdd-4ba1-93d0-d7c5b24535ea None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:08:48.624383 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8366a659-1bdd-4ba1-93d0-d7c5b24535ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:48.769076 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8366a659-1bdd-4ba1-93d0-d7c5b24535ea None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:08:48.795506 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:48.890674 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf295d67-1561-4cdf-a503-6c4dcb92fead service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:08:48.891322 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf295d67-1561-4cdf-a503-6c4dcb92fead service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:48.891636 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf295d67-1561-4cdf-a503-6c4dcb92fead service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:48.891913 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf295d67-1561-4cdf-a503-6c4dcb92fead service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:48.893564 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-cf295d67-1561-4cdf-a503-6c4dcb92fead service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:48.984384 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf295d67-1561-4cdf-a503-6c4dcb92fead service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:08:48.986005 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf295d67-1561-4cdf-a503-6c4dcb92fead service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:08:49.004934 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 723/1440] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:08:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:08:50.957240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356233fe-b643-445c-ad1e-005a15956c1c None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:50.957768 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356233fe-b643-445c-ad1e-005a15956c1c None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:50.957768 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356233fe-b643-445c-ad1e-005a15956c1c None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:50.959100 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 718/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 23:08:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:08:50.965070 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0bab79-a822-4901-ab80-d4362c2ce6e8 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:50.965532 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0bab79-a822-4901-ab80-d4362c2ce6e8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:50.965625 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0bab79-a822-4901-ab80-d4362c2ce6e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:51.005471 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:51.031735 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-1f0bab79-a822-4901-ab80-d4362c2ce6e8 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:08:51.032917 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1f0bab79-a822-4901-ab80-d4362c2ce6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:51.131864 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 724/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 23:08:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:08:51.171102 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356233fe-b643-445c-ad1e-005a15956c1c None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:08:51.172685 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-356233fe-b643-445c-ad1e-005a15956c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:51.280059 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356233fe-b643-445c-ad1e-005a15956c1c None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:08:51.311029 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:51.353941 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58106ff-6cc0-4e1d-8888-f1342e74a897 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:08:51.354612 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58106ff-6cc0-4e1d-8888-f1342e74a897 service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:51.354898 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58106ff-6cc0-4e1d-8888-f1342e74a897 service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:51.355173 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58106ff-6cc0-4e1d-8888-f1342e74a897 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:51.356980 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-b58106ff-6cc0-4e1d-8888-f1342e74a897 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:51.469231 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58106ff-6cc0-4e1d-8888-f1342e74a897 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:08:51.470700 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58106ff-6cc0-4e1d-8888-f1342e74a897 service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:08:51.489003 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 719/1443] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:08:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:08:53.738863 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1500b7-5b4d-4588-a682-b019a04fe31b None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:53.739782 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1500b7-5b4d-4588-a682-b019a04fe31b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:53.739974 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1500b7-5b4d-4588-a682-b019a04fe31b None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:53.741444 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 725/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 23:08:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:08:53.748286 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a654f5ab-7d03-4f15-b5a2-f8ef8632c915 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:53.748833 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a654f5ab-7d03-4f15-b5a2-f8ef8632c915 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:53.748833 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a654f5ab-7d03-4f15-b5a2-f8ef8632c915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:53.826207 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-a654f5ab-7d03-4f15-b5a2-f8ef8632c915 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:08:53.828378 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-a654f5ab-7d03-4f15-b5a2-f8ef8632c915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:53.852985 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:53.952361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 720/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 23:08:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:08:53.993966 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1500b7-5b4d-4588-a682-b019a04fe31b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:08:53.995546 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-cf1500b7-5b4d-4588-a682-b019a04fe31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:54.129463 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1500b7-5b4d-4588-a682-b019a04fe31b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:08:54.143812 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:54.221781 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf8391-416d-4cc9-aba2-5da0baf7af67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:08:54.222429 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf8391-416d-4cc9-aba2-5da0baf7af67 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:54.222710 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf8391-416d-4cc9-aba2-5da0baf7af67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:54.222981 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf8391-416d-4cc9-aba2-5da0baf7af67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:54.224700 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a9bf8391-416d-4cc9-aba2-5da0baf7af67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:54.347672 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bf8391-416d-4cc9-aba2-5da0baf7af67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:08:54.349046 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bf8391-416d-4cc9-aba2-5da0baf7af67 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:08:54.378208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 726/1446] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:08:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:08:56.799779 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5236fe1-1363-4393-abb1-c92db5cf3f70 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:56.800344 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5236fe1-1363-4393-abb1-c92db5cf3f70 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:56.800344 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5236fe1-1363-4393-abb1-c92db5cf3f70 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:56.801758 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 721/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 23:08:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:08:56.808043 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc823e-6031-4063-924a-65981f9f685b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:56.808554 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc823e-6031-4063-924a-65981f9f685b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:56.808660 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc823e-6031-4063-924a-65981f9f685b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:56.859433 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:56.878481 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-78dc823e-6031-4063-924a-65981f9f685b None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:08:56.879754 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-78dc823e-6031-4063-924a-65981f9f685b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:56.976156 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 727/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 23:08:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:08:57.015215 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5236fe1-1363-4393-abb1-c92db5cf3f70 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:08:57.016858 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-f5236fe1-1363-4393-abb1-c92db5cf3f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:57.123768 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5236fe1-1363-4393-abb1-c92db5cf3f70 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:08:57.163455 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:57.199874 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e2a279-f459-4f5b-a6d8-307def1d09b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:08:57.200609 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2a279-f459-4f5b-a6d8-307def1d09b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:57.200894 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2a279-f459-4f5b-a6d8-307def1d09b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:57.201167 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2a279-f459-4f5b-a6d8-307def1d09b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:57.202833 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-51e2a279-f459-4f5b-a6d8-307def1d09b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:57.312566 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e2a279-f459-4f5b-a6d8-307def1d09b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:08:57.314210 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e2a279-f459-4f5b-a6d8-307def1d09b5 service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:08:57.332471 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 722/1449] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:08:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:08:59.555059 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5c802-6dfc-40ed-8ce0-a556c50e1258 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:59.556210 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5c802-6dfc-40ed-8ce0-a556c50e1258 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:59.556426 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5c802-6dfc-40ed-8ce0-a556c50e1258 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:59.558775 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 728/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 23:08:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:08:59.565460 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aaefb0-5aee-46fd-8ff7-f0a435b11369 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:59.566002 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aaefb0-5aee-46fd-8ff7-f0a435b11369 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:59.566002 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aaefb0-5aee-46fd-8ff7-f0a435b11369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:59.635826 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-58aaefb0-5aee-46fd-8ff7-f0a435b11369 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:08:59.637072 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-58aaefb0-5aee-46fd-8ff7-f0a435b11369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:59.658543 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:59.739215 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 723/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 23:08:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:08:59.779220 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada5c802-6dfc-40ed-8ce0-a556c50e1258 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:08:59.780799 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ada5c802-6dfc-40ed-8ce0-a556c50e1258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:08:59.889095 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada5c802-6dfc-40ed-8ce0-a556c50e1258 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:08:59.909591 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:08:59.972167 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c70276-9f88-4792-b226-c37d02df280c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:08:59.972824 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c70276-9f88-4792-b226-c37d02df280c service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:08:59.973103 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c70276-9f88-4792-b226-c37d02df280c service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:08:59.973370 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c70276-9f88-4792-b226-c37d02df280c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:08:59.975091 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d0c70276-9f88-4792-b226-c37d02df280c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:00.103629 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c70276-9f88-4792-b226-c37d02df280c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:09:00.105085 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c70276-9f88-4792-b226-c37d02df280c service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:09:00.125375 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 729/1452] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:08:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:09:02.249693 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab4d08-5dfa-4e4a-b8b0-654aeab7776e None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:02.250273 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab4d08-5dfa-4e4a-b8b0-654aeab7776e None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:02.250273 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab4d08-5dfa-4e4a-b8b0-654aeab7776e None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:02.251691 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 724/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 23:09:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:09:02.257787 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8fa7dd-c5e0-4cee-8a56-3b5dad554e95 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:02.258872 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8fa7dd-c5e0-4cee-8a56-3b5dad554e95 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:02.258872 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8fa7dd-c5e0-4cee-8a56-3b5dad554e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:02.323019 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:09:02.326102 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-6a8fa7dd-c5e0-4cee-8a56-3b5dad554e95 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:09:02.327443 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6a8fa7dd-c5e0-4cee-8a56-3b5dad554e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:02.435554 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 730/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 23:09:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:09:02.473102 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab4d08-5dfa-4e4a-b8b0-654aeab7776e None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:09:02.474692 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-56ab4d08-5dfa-4e4a-b8b0-654aeab7776e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:02.615681 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab4d08-5dfa-4e4a-b8b0-654aeab7776e None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:09:02.674254 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:09:02.707547 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147fda29-d86c-44ea-85b1-f47a4481247b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:09:02.708355 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147fda29-d86c-44ea-85b1-f47a4481247b service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:02.708639 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147fda29-d86c-44ea-85b1-f47a4481247b service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:02.708910 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147fda29-d86c-44ea-85b1-f47a4481247b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:02.710708 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-147fda29-d86c-44ea-85b1-f47a4481247b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:02.840353 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147fda29-d86c-44ea-85b1-f47a4481247b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:09:02.841877 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147fda29-d86c-44ea-85b1-f47a4481247b service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:09:02.863244 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 725/1455] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:09:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:09:05.029752 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad16a6-2276-4f1d-ba52-a0cec09e2375 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:05.030273 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad16a6-2276-4f1d-ba52-a0cec09e2375 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:05.030273 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad16a6-2276-4f1d-ba52-a0cec09e2375 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:05.031671 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 731/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 23:09:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:09:05.037312 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e888d1-df0e-4dd0-8f4b-a34280372404 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:05.037860 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e888d1-df0e-4dd0-8f4b-a34280372404 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:05.037860 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e888d1-df0e-4dd0-8f4b-a34280372404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:05.107494 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-44e888d1-df0e-4dd0-8f4b-a34280372404 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:09:05.108772 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-44e888d1-df0e-4dd0-8f4b-a34280372404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:05.140165 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:09:05.214161 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 726/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 23:09:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:09:05.252737 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad16a6-2276-4f1d-ba52-a0cec09e2375 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:09:05.254290 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-daad16a6-2276-4f1d-ba52-a0cec09e2375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:05.366217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad16a6-2276-4f1d-ba52-a0cec09e2375 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:09:05.396299 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:09:05.448922 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b744a1a3-9f0d-4d3c-91ff-ea3d78c97f5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:09:05.449570 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744a1a3-9f0d-4d3c-91ff-ea3d78c97f5d service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:05.449851 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744a1a3-9f0d-4d3c-91ff-ea3d78c97f5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:05.450121 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744a1a3-9f0d-4d3c-91ff-ea3d78c97f5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:05.451829 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b744a1a3-9f0d-4d3c-91ff-ea3d78c97f5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:05.570610 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b744a1a3-9f0d-4d3c-91ff-ea3d78c97f5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:09:05.572128 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b744a1a3-9f0d-4d3c-91ff-ea3d78c97f5d service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:09:05.592971 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 732/1458] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:09:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:09:07.719844 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47213cb9-2f31-4def-ba8e-63f1a6e81081 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:07.720455 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47213cb9-2f31-4def-ba8e-63f1a6e81081 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:07.720455 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47213cb9-2f31-4def-ba8e-63f1a6e81081 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:07.721794 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 727/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 23:09:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:09:07.727772 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe952742-901a-4489-bda6-8c29ae2f9cfe None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:07.728569 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe952742-901a-4489-bda6-8c29ae2f9cfe None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:07.728742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe952742-901a-4489-bda6-8c29ae2f9cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:07.806434 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-fe952742-901a-4489-bda6-8c29ae2f9cfe None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:09:07.807712 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-fe952742-901a-4489-bda6-8c29ae2f9cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:07.816974 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:09:07.965606 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 733/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 23:09:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:09:08.004486 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47213cb9-2f31-4def-ba8e-63f1a6e81081 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:09:08.006029 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-47213cb9-2f31-4def-ba8e-63f1a6e81081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:08.125145 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47213cb9-2f31-4def-ba8e-63f1a6e81081 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:09:08.186901 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:09:08.207199 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca2ca9-331c-4db8-b43b-54eb4d0decab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:09:08.207892 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca2ca9-331c-4db8-b43b-54eb4d0decab service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:08.208178 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca2ca9-331c-4db8-b43b-54eb4d0decab service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:08.208456 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca2ca9-331c-4db8-b43b-54eb4d0decab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:08.210169 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-8dca2ca9-331c-4db8-b43b-54eb4d0decab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:08.330802 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dca2ca9-331c-4db8-b43b-54eb4d0decab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:09:08.332441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dca2ca9-331c-4db8-b43b-54eb4d0decab service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:09:08.353928 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 728/1461] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:09:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:09:10.880830 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae2738-0f56-4632-bbdc-0aee5b815c30 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:10.881395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae2738-0f56-4632-bbdc-0aee5b815c30 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:10.881395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae2738-0f56-4632-bbdc-0aee5b815c30 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:10.882924 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 734/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 23:09:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:09:10.889417 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d85f08-da4d-46eb-9651-7c7acd7d6672 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:10.890208 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d85f08-da4d-46eb-9651-7c7acd7d6672 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:10.890398 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d85f08-da4d-46eb-9651-7c7acd7d6672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:10.963468 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-88d85f08-da4d-46eb-9651-7c7acd7d6672 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:09:10.964869 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-88d85f08-da4d-46eb-9651-7c7acd7d6672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:11.008256 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:09:11.078531 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 729/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 23:09:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:09:11.122523 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae2738-0f56-4632-bbdc-0aee5b815c30 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:09:11.124766 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c5ae2738-0f56-4632-bbdc-0aee5b815c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:11.244569 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae2738-0f56-4632-bbdc-0aee5b815c30 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:09:11.286324 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:09:11.328555 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121284b3-5217-4e7b-985f-a7d5f54e8d63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:09:11.329215 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121284b3-5217-4e7b-985f-a7d5f54e8d63 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:11.329498 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121284b3-5217-4e7b-985f-a7d5f54e8d63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:11.329773 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121284b3-5217-4e7b-985f-a7d5f54e8d63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:11.331467 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-121284b3-5217-4e7b-985f-a7d5f54e8d63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:11.458134 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121284b3-5217-4e7b-985f-a7d5f54e8d63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:09:11.459643 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121284b3-5217-4e7b-985f-a7d5f54e8d63 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:09:11.481772 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 735/1464] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:09:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:09:13.657301 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a442eda-229d-4fb5-bd35-3d982020852c None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:13.657959 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a442eda-229d-4fb5-bd35-3d982020852c None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:13.657959 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a442eda-229d-4fb5-bd35-3d982020852c None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:13.659267 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 730/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 23:09:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:09:13.665366 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92afcc2-cc4b-4254-ab79-96be5cdd2289 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:13.665822 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92afcc2-cc4b-4254-ab79-96be5cdd2289 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:13.665924 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92afcc2-cc4b-4254-ab79-96be5cdd2289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:13.731132 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-f92afcc2-cc4b-4254-ab79-96be5cdd2289 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:09:13.732387 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f92afcc2-cc4b-4254-ab79-96be5cdd2289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:13.750748 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:09:13.843418 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 736/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 23:09:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:09:13.880087 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a442eda-229d-4fb5-bd35-3d982020852c None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:09:13.881649 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-4a442eda-229d-4fb5-bd35-3d982020852c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:13.993475 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a442eda-229d-4fb5-bd35-3d982020852c None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:09:14.062490 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:09:14.069939 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867fccdd-380b-4caa-a48b-0b178b50718c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:09:14.070860 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867fccdd-380b-4caa-a48b-0b178b50718c service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:14.071161 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867fccdd-380b-4caa-a48b-0b178b50718c service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:14.071435 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867fccdd-380b-4caa-a48b-0b178b50718c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:14.073496 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-867fccdd-380b-4caa-a48b-0b178b50718c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:14.189461 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867fccdd-380b-4caa-a48b-0b178b50718c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:09:14.190932 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867fccdd-380b-4caa-a48b-0b178b50718c service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:09:14.209471 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 731/1467] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:09:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:09:16.277833 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f80f9-d536-49d4-80e4-a1dbe8d25707 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:16.278426 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f80f9-d536-49d4-80e4-a1dbe8d25707 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:16.278426 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f80f9-d536-49d4-80e4-a1dbe8d25707 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:16.279876 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 737/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 23:09:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:09:16.287329 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307e151-a48d-4ba6-859e-1fe200b8cfe3 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:16.287885 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307e151-a48d-4ba6-859e-1fe200b8cfe3 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:16.288084 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307e151-a48d-4ba6-859e-1fe200b8cfe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:16.361273 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-3307e151-a48d-4ba6-859e-1fe200b8cfe3 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:09:16.362546 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-3307e151-a48d-4ba6-859e-1fe200b8cfe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:16.416860 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:09:16.468925 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 732/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 23:09:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:09:16.505778 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f80f9-d536-49d4-80e4-a1dbe8d25707 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:09:16.507332 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-371f80f9-d536-49d4-80e4-a1dbe8d25707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:16.632643 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f80f9-d536-49d4-80e4-a1dbe8d25707 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:09:16.685207 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:09:16.717157 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8482aff8-4d5e-48af-a0ff-ace0b7a343af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:09:16.717630 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8482aff8-4d5e-48af-a0ff-ace0b7a343af service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:16.717911 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8482aff8-4d5e-48af-a0ff-ace0b7a343af service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:16.718183 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8482aff8-4d5e-48af-a0ff-ace0b7a343af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:16.719942 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8482aff8-4d5e-48af-a0ff-ace0b7a343af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:16.835841 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8482aff8-4d5e-48af-a0ff-ace0b7a343af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:09:16.837940 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8482aff8-4d5e-48af-a0ff-ace0b7a343af service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:09:16.859042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 738/1470] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:09:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:09:26.466930 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=22 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:09:26.856569 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=66 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:09:37.019137 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da887b89-c26b-41ac-ab26-543420385724 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:37.019775 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da887b89-c26b-41ac-ab26-543420385724 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:37.019775 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da887b89-c26b-41ac-ab26-543420385724 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:37.021039 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 733/1471] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:09:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:09:37.026851 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59abdb8e-8cef-47a5-8197-62cc5093dae3 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:37.027350 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59abdb8e-8cef-47a5-8197-62cc5093dae3 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:37.027481 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59abdb8e-8cef-47a5-8197-62cc5093dae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:37.097910 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-59abdb8e-8cef-47a5-8197-62cc5093dae3 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:09:37.099228 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-59abdb8e-8cef-47a5-8197-62cc5093dae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:37.204648 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 739/1472] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:09:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:09:37.292545 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6ce91-de28-45bc-882b-3ca124025545 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:37.292977 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6ce91-de28-45bc-882b-3ca124025545 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:37.293313 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6ce91-de28-45bc-882b-3ca124025545 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:37.295008 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 734/1473] 10.222.0.60 () {54 vars in 962 bytes} [Wed Jul 13 23:09:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:09:37.302309 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af550e-f695-4db3-a64c-55b07fcc11ed None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:37.302790 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af550e-f695-4db3-a64c-55b07fcc11ed None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:37.303232 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af550e-f695-4db3-a64c-55b07fcc11ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:37.386245 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-a9af550e-f695-4db3-a64c-55b07fcc11ed None None] MFA Rules not processed for user `2ce9e6497fdb4068b23aa3f7b2c19536`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:09:37.387487 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a9af550e-f695-4db3-a64c-55b07fcc11ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:37.410291 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:09:37.467904 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 740/1474] 10.222.0.60 () {58 vars in 1059 bytes} [Wed Jul 13 23:09:37 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:09:37.475357 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6ce91-de28-45bc-882b-3ca124025545 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:09:37.477606 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-d4b6ce91-de28-45bc-882b-3ca124025545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:37.585974 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6ce91-de28-45bc-882b-3ca124025545 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:09:37.663358 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a3bc38-e6c4-4bfd-8ed5-bcd44113b874 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ce9e6497fdb4068b23aa3f7b2c19536', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:09:37.664074 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3bc38-e6c4-4bfd-8ed5-bcd44113b874 service glance] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:37.664365 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3bc38-e6c4-4bfd-8ed5-bcd44113b874 service glance] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:37.664645 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3bc38-e6c4-4bfd-8ed5-bcd44113b874 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:37.666392 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-91a3bc38-e6c4-4bfd-8ed5-bcd44113b874 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:37.724034 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:09:37.795070 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a3bc38-e6c4-4bfd-8ed5-bcd44113b874 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:09:37.797427 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a3bc38-e6c4-4bfd-8ed5-bcd44113b874 service glance] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:09:37.824914 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 735/1475] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:09:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:09:40.111308 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267ef82-37a0-4d54-9be7-8a8572090d99 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:40.112240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267ef82-37a0-4d54-9be7-8a8572090d99 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:40.112240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267ef82-37a0-4d54-9be7-8a8572090d99 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:40.113314 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 741/1476] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:09:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:09:40.119709 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7bf03-ba5b-41e3-a535-5d98333ca99d None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:40.120186 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7bf03-ba5b-41e3-a535-5d98333ca99d None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:40.120322 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7bf03-ba5b-41e3-a535-5d98333ca99d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:40.191399 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-42b7bf03-ba5b-41e3-a535-5d98333ca99d None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:09:40.192723 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-42b7bf03-ba5b-41e3-a535-5d98333ca99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:40.294051 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 736/1477] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:09:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:09:40.317465 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f267ef82-37a0-4d54-9be7-8a8572090d99 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:09:40.319014 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f267ef82-37a0-4d54-9be7-8a8572090d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:40.466199 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f267ef82-37a0-4d54-9be7-8a8572090d99 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:09:40.517954 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:09:40.565263 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23485ea3-af4e-47d6-8d40-fcf66e4781c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:09:40.565916 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23485ea3-af4e-47d6-8d40-fcf66e4781c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:40.566194 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23485ea3-af4e-47d6-8d40-fcf66e4781c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:40.566465 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23485ea3-af4e-47d6-8d40-fcf66e4781c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:40.567194 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23485ea3-af4e-47d6-8d40-fcf66e4781c1 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:09:40.569044 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: WARNING py.warnings [None req-23485ea3-af4e-47d6-8d40-fcf66e4781c1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 23:09:40.569044 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: warnings.warn(msg) Jul 13 23:09:40.569044 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: Jul 13 23:09:40.570729 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23485ea3-af4e-47d6-8d40-fcf66e4781c1 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:09:40.590102 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 742/1478] 10.222.0.60 () {56 vars in 1108 bytes} [Wed Jul 13 23:09:40 2022] GET /identity/v3/projects => generated 2553 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:09:42.701257 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0112310-368a-482f-8fb1-ac3b6e0f31d7 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:42.701823 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0112310-368a-482f-8fb1-ac3b6e0f31d7 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:42.701823 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0112310-368a-482f-8fb1-ac3b6e0f31d7 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:42.703268 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 737/1479] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:09:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:09:42.709245 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb0f6dc-a9bf-49ad-80cc-d762479d47c2 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:42.709825 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb0f6dc-a9bf-49ad-80cc-d762479d47c2 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:42.709825 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb0f6dc-a9bf-49ad-80cc-d762479d47c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:42.903010 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-cdb0f6dc-a9bf-49ad-80cc-d762479d47c2 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:09:42.904292 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-cdb0f6dc-a9bf-49ad-80cc-d762479d47c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:43.004546 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 743/1480] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:09:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:09:43.049555 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0112310-368a-482f-8fb1-ac3b6e0f31d7 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:09:43.051204 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-d0112310-368a-482f-8fb1-ac3b6e0f31d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:43.061554 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:09:43.211246 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0112310-368a-482f-8fb1-ac3b6e0f31d7 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:09:43.295252 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:09:43.330807 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8546cabd-e0fb-4841-856e-9eb1aec7c464 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:09:43.331465 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546cabd-e0fb-4841-856e-9eb1aec7c464 service nova] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:43.331797 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546cabd-e0fb-4841-856e-9eb1aec7c464 service nova] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:43.332065 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546cabd-e0fb-4841-856e-9eb1aec7c464 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:43.333733 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-8546cabd-e0fb-4841-856e-9eb1aec7c464 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:43.463004 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8546cabd-e0fb-4841-856e-9eb1aec7c464 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:09:43.464677 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8546cabd-e0fb-4841-856e-9eb1aec7c464 service nova] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:09:43.485460 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 738/1481] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:09:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:09:45.750358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ceb588-d2f3-4675-9e03-c6ab33580f20 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:45.751000 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ceb588-d2f3-4675-9e03-c6ab33580f20 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:45.751000 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ceb588-d2f3-4675-9e03-c6ab33580f20 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:45.752470 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 744/1482] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:09:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:09:45.760610 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f45be-9594-4a0b-9ad6-a893f2b1497a None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:45.761188 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f45be-9594-4a0b-9ad6-a893f2b1497a None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:45.761188 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f45be-9594-4a0b-9ad6-a893f2b1497a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:45.840676 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-df9f45be-9594-4a0b-9ad6-a893f2b1497a None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:09:45.841913 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-df9f45be-9594-4a0b-9ad6-a893f2b1497a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:45.895481 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:09:45.952491 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 739/1483] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:09:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:09:45.988559 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ceb588-d2f3-4675-9e03-c6ab33580f20 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:09:45.990150 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-31ceb588-d2f3-4675-9e03-c6ab33580f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:46.014273 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:09:46.107307 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ceb588-d2f3-4675-9e03-c6ab33580f20 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:09:46.185350 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a5d67f-94b4-41b6-8268-b5205bd1e7c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:09:46.185993 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a5d67f-94b4-41b6-8268-b5205bd1e7c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:46.186269 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a5d67f-94b4-41b6-8268-b5205bd1e7c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:46.186536 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a5d67f-94b4-41b6-8268-b5205bd1e7c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:46.188215 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c8a5d67f-94b4-41b6-8268-b5205bd1e7c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:46.272225 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:09:46.309266 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a5d67f-94b4-41b6-8268-b5205bd1e7c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:09:46.310703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a5d67f-94b4-41b6-8268-b5205bd1e7c5 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:09:46.329157 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 745/1484] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:09:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:09:48.528193 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd9e91-f3b7-4286-a542-20079837beea None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:48.528784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd9e91-f3b7-4286-a542-20079837beea None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:48.528784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd9e91-f3b7-4286-a542-20079837beea None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:48.530123 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 740/1485] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:09:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:09:48.535849 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577b010-0173-49cd-a430-5c03cbcfa89d None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:48.536751 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577b010-0173-49cd-a430-5c03cbcfa89d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:48.536751 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577b010-0173-49cd-a430-5c03cbcfa89d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:48.609884 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-1577b010-0173-49cd-a430-5c03cbcfa89d None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:09:48.611210 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1577b010-0173-49cd-a430-5c03cbcfa89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:48.721679 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 746/1486] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:09:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:09:48.761797 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd9e91-f3b7-4286-a542-20079837beea None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:09:48.764775 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-f5bd9e91-f3b7-4286-a542-20079837beea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:48.928802 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd9e91-f3b7-4286-a542-20079837beea None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:09:49.013815 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:09:49.016209 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b66aaf-0249-45f1-9921-d44682ab5ca6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:09:49.017260 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b66aaf-0249-45f1-9921-d44682ab5ca6 service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:49.017674 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b66aaf-0249-45f1-9921-d44682ab5ca6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:49.018056 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b66aaf-0249-45f1-9921-d44682ab5ca6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:49.020688 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-89b66aaf-0249-45f1-9921-d44682ab5ca6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:49.141815 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b66aaf-0249-45f1-9921-d44682ab5ca6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:09:49.143371 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b66aaf-0249-45f1-9921-d44682ab5ca6 service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:09:49.164114 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 741/1487] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:09:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:09:51.372971 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3640a7-af66-4d80-a726-3f73452aa95c None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:51.373492 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3640a7-af66-4d80-a726-3f73452aa95c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:51.373602 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3640a7-af66-4d80-a726-3f73452aa95c None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:51.374922 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 747/1488] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:09:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:09:51.381134 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e19cc4-c6a7-43d2-88a3-477f37b5c6ac None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:51.381586 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e19cc4-c6a7-43d2-88a3-477f37b5c6ac None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:51.381730 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e19cc4-c6a7-43d2-88a3-477f37b5c6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:51.449363 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-71e19cc4-c6a7-43d2-88a3-477f37b5c6ac None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:09:51.450627 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-71e19cc4-c6a7-43d2-88a3-477f37b5c6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:51.505191 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:09:51.563114 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 742/1489] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:09:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:09:51.605170 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3640a7-af66-4d80-a726-3f73452aa95c None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:09:51.607095 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-2b3640a7-af66-4d80-a726-3f73452aa95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:51.644260 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:09:51.720114 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3640a7-af66-4d80-a726-3f73452aa95c None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:09:51.795742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de4c02-9f0c-4421-a150-bff3c60b9886 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:09:51.796386 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de4c02-9f0c-4421-a150-bff3c60b9886 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:51.796665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de4c02-9f0c-4421-a150-bff3c60b9886 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:51.796936 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de4c02-9f0c-4421-a150-bff3c60b9886 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:51.798583 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c7de4c02-9f0c-4421-a150-bff3c60b9886 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:51.888461 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:09:51.944053 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de4c02-9f0c-4421-a150-bff3c60b9886 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:09:51.946880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de4c02-9f0c-4421-a150-bff3c60b9886 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:09:51.980792 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 748/1490] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:09:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:09:54.248086 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b0c5e-042d-454e-921f-afaeaf0e827e None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:54.248849 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b0c5e-042d-454e-921f-afaeaf0e827e None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:54.248849 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b0c5e-042d-454e-921f-afaeaf0e827e None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:54.250001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 743/1491] 10.222.0.60 () {54 vars in 926 bytes} [Wed Jul 13 23:09:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:09:54.256535 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd63b6-4cc1-40ca-a687-b3582dba5a63 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:54.257602 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd63b6-4cc1-40ca-a687-b3582dba5a63 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:54.257763 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd63b6-4cc1-40ca-a687-b3582dba5a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:54.324585 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-b9dd63b6-4cc1-40ca-a687-b3582dba5a63 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:09:54.325808 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b9dd63b6-4cc1-40ca-a687-b3582dba5a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:54.441358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 749/1492] 10.222.0.60 () {58 vars in 1023 bytes} [Wed Jul 13 23:09:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:09:54.481939 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b0c5e-042d-454e-921f-afaeaf0e827e None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:09:54.483476 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-8b1b0c5e-042d-454e-921f-afaeaf0e827e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:54.605161 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b0c5e-042d-454e-921f-afaeaf0e827e None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:09:54.685931 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbbe993-282c-4d29-b332-8328c7d50d8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:09:54.686896 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbe993-282c-4d29-b332-8328c7d50d8a service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:09:54.687134 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbe993-282c-4d29-b332-8328c7d50d8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:09:54.687515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbe993-282c-4d29-b332-8328c7d50d8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:09:54.690126 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-2bbbe993-282c-4d29-b332-8328c7d50d8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:09:54.703679 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:09:54.813805 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbbe993-282c-4d29-b332-8328c7d50d8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:09:54.815281 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbbe993-282c-4d29-b332-8328c7d50d8a service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:09:54.834874 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 744/1493] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:09:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:10:04.439416 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=82 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:10:04.832284 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=50 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:11:31.180488 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bfee3a-b048-4650-b5ef-51e2f85e2c04 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:11:31.181120 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bfee3a-b048-4650-b5ef-51e2f85e2c04 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:11:31.181120 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bfee3a-b048-4650-b5ef-51e2f85e2c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:11:31.253854 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-56bfee3a-b048-4650-b5ef-51e2f85e2c04 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:11:31.255578 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-56bfee3a-b048-4650-b5ef-51e2f85e2c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:11:31.350294 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 750/1494] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:11:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:11:31.358576 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72029832-906e-4c7a-bb32-789206efe0d2 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:11:31.359130 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72029832-906e-4c7a-bb32-789206efe0d2 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:11:31.359130 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72029832-906e-4c7a-bb32-789206efe0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:11:31.436020 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-72029832-906e-4c7a-bb32-789206efe0d2 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:11:31.437414 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-72029832-906e-4c7a-bb32-789206efe0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:11:31.534536 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 745/1495] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:11:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:11:31.539375 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bfee3a-b048-4650-b5ef-51e2f85e2c04 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:11:31.540646 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-56bfee3a-b048-4650-b5ef-51e2f85e2c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:11:31.552867 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bfee3a-b048-4650-b5ef-51e2f85e2c04 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:11:31.612555 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:11:31.647710 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428238a-97e2-486c-b52f-844b920e4ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:11:31.648373 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428238a-97e2-486c-b52f-844b920e4ed7 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:11:31.648660 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428238a-97e2-486c-b52f-844b920e4ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:11:31.648932 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428238a-97e2-486c-b52f-844b920e4ed7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:11:31.649654 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a428238a-97e2-486c-b52f-844b920e4ed7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:11:31.649958 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a428238a-97e2-486c-b52f-844b920e4ed7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:11:31.650975 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: WARNING py.warnings [None req-a428238a-97e2-486c-b52f-844b920e4ed7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 13 23:11:31.650975 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: warnings.warn(msg) Jul 13 23:11:31.650975 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: Jul 13 23:11:31.651890 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a428238a-97e2-486c-b52f-844b920e4ed7 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:11:31.663231 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 751/1496] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:11:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:11:31.671646 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72029832-906e-4c7a-bb32-789206efe0d2 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:11:31.681290 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72029832-906e-4c7a-bb32-789206efe0d2 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:11:31.745562 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:11:31.781001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c72c6-7e4a-49e9-8dbe-ef9ca2c3db65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:11:31.781668 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c72c6-7e4a-49e9-8dbe-ef9ca2c3db65 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:11:31.781948 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c72c6-7e4a-49e9-8dbe-ef9ca2c3db65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:11:31.782258 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c72c6-7e4a-49e9-8dbe-ef9ca2c3db65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:11:31.782943 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c72c6-7e4a-49e9-8dbe-ef9ca2c3db65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:11:31.784799 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: WARNING py.warnings [None req-8c5c72c6-7e4a-49e9-8dbe-ef9ca2c3db65 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 23:11:31.784799 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: warnings.warn(msg) Jul 13 23:11:31.784799 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: Jul 13 23:11:31.786019 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c72c6-7e4a-49e9-8dbe-ef9ca2c3db65 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:11:31.840814 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 746/1497] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:11:31 2022] POST /identity/v3/projects => generated 377 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:11:31.844881 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428238a-97e2-486c-b52f-844b920e4ed7 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:11:31.846408 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a428238a-97e2-486c-b52f-844b920e4ed7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:11:31.879056 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428238a-97e2-486c-b52f-844b920e4ed7 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:11:31.975316 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd31b173-8a63-4e9a-8818-86369c84e8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:11:31.976025 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd31b173-8a63-4e9a-8818-86369c84e8f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:11:31.976307 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd31b173-8a63-4e9a-8818-86369c84e8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:11:31.976582 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd31b173-8a63-4e9a-8818-86369c84e8f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:11:31.977294 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd31b173-8a63-4e9a-8818-86369c84e8f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:11:31.978324 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: WARNING py.warnings [None req-dd31b173-8a63-4e9a-8818-86369c84e8f6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 23:11:31.978324 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: warnings.warn(msg) Jul 13 23:11:31.978324 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: Jul 13 23:11:31.979650 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd31b173-8a63-4e9a-8818-86369c84e8f6 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:11:32.024215 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 752/1498] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:11:31 2022] POST /identity/v3/users => generated 289 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:11:32.028316 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c72c6-7e4a-49e9-8dbe-ef9ca2c3db65 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:11:32.029846 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-8c5c72c6-7e4a-49e9-8dbe-ef9ca2c3db65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:11:32.092365 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c72c6-7e4a-49e9-8dbe-ef9ca2c3db65 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:11:32.178626 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:11:32.190522 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02570a5-49fe-4a32-aa5d-f3b0d0987ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:11:32.191127 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02570a5-49fe-4a32-aa5d-f3b0d0987ecd admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:11:32.191408 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02570a5-49fe-4a32-aa5d-f3b0d0987ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:11:32.191734 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02570a5-49fe-4a32-aa5d-f3b0d0987ecd admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:11:32.192495 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02570a5-49fe-4a32-aa5d-f3b0d0987ecd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:11:32.193559 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: WARNING py.warnings [None req-d02570a5-49fe-4a32-aa5d-f3b0d0987ecd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 13 23:11:32.193559 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: warnings.warn(msg) Jul 13 23:11:32.193559 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: Jul 13 23:11:32.194487 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02570a5-49fe-4a32-aa5d-f3b0d0987ecd admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:11:32.207002 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 747/1499] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:11:32 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:11:32.211352 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd31b173-8a63-4e9a-8818-86369c84e8f6 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:11:32.212917 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-dd31b173-8a63-4e9a-8818-86369c84e8f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:11:32.226258 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd31b173-8a63-4e9a-8818-86369c84e8f6 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:11:32.253423 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:11:32.329200 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02729a4a-5d21-4b31-94f4-d34a050fdfd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:11:32.329765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02729a4a-5d21-4b31-94f4-d34a050fdfd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:11:32.330044 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02729a4a-5d21-4b31-94f4-d34a050fdfd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:11:32.330313 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02729a4a-5d21-4b31-94f4-d34a050fdfd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:11:32.331044 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02729a4a-5d21-4b31-94f4-d34a050fdfd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:11:32.332104 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: WARNING py.warnings [None req-02729a4a-5d21-4b31-94f4-d34a050fdfd2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 13 23:11:32.332104 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: warnings.warn(msg) Jul 13 23:11:32.332104 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: Jul 13 23:11:32.333489 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02729a4a-5d21-4b31-94f4-d34a050fdfd2 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:11:32.346601 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 753/1500] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:11:32 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:11:32.351481 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02570a5-49fe-4a32-aa5d-f3b0d0987ecd admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:11:32.353195 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-d02570a5-49fe-4a32-aa5d-f3b0d0987ecd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:11:32.389101 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02570a5-49fe-4a32-aa5d-f3b0d0987ecd admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:11:32.487488 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325cf440-0199-4195-8749-dc36795a28ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:11:32.488414 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325cf440-0199-4195-8749-dc36795a28ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:11:32.488730 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325cf440-0199-4195-8749-dc36795a28ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:11:32.489004 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325cf440-0199-4195-8749-dc36795a28ba admin admin] PATH_INFO: `/v3/projects/8b0032952cf647a985c7b085e0afff08/users/64169e27988c4c4d8449c48427b8419b/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:11:32.511416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325cf440-0199-4195-8749-dc36795a28ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b0032952cf647a985c7b085e0afff08, user_id=64169e27988c4c4d8449c48427b8419b, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:11:32.512502 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: WARNING py.warnings [None req-325cf440-0199-4195-8749-dc36795a28ba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 23:11:32.512502 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: warnings.warn(msg) Jul 13 23:11:32.512502 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: Jul 13 23:11:32.514434 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325cf440-0199-4195-8749-dc36795a28ba admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:11:32.541205 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 748/1501] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:11:32 2022] PUT /identity/v3/projects/8b0032952cf647a985c7b085e0afff08/users/64169e27988c4c4d8449c48427b8419b/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:11:32.554106 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbbf533-85d3-4283-b47c-6887141af801 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:11:32.554389 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbbf533-85d3-4283-b47c-6887141af801 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:11:32.554677 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbbf533-85d3-4283-b47c-6887141af801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:11:32.611643 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-cdbbf533-85d3-4283-b47c-6887141af801 None None] MFA Rules not processed for user `64169e27988c4c4d8449c48427b8419b`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:11:32.612798 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-cdbbf533-85d3-4283-b47c-6887141af801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:11:32.697864 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 754/1502] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:11:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:11:32.711733 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c0ce7-c77b-4f8e-932e-83995be75c37 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:11:32.712226 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c0ce7-c77b-4f8e-932e-83995be75c37 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:11:32.712641 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c0ce7-c77b-4f8e-932e-83995be75c37 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:11:32.715114 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 749/1503] 10.222.0.60 () {54 vars in 963 bytes} [Wed Jul 13 23:11:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:11:32.721256 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1ae95-0cc5-4b0c-bc3c-b12959da8f82 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:11:32.721575 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1ae95-0cc5-4b0c-bc3c-b12959da8f82 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:11:32.721980 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1ae95-0cc5-4b0c-bc3c-b12959da8f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:11:32.753237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:11:32.797520 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-e9c1ae95-0cc5-4b0c-bc3c-b12959da8f82 None None] MFA Rules not processed for user `54661bad914f455cb00d42ec692fde11`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:11:32.798745 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e9c1ae95-0cc5-4b0c-bc3c-b12959da8f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:11:32.877380 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 755/1504] 10.222.0.60 () {58 vars in 1060 bytes} [Wed Jul 13 23:11:32 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:11:32.884000 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c0ce7-c77b-4f8e-932e-83995be75c37 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:11:32.885562 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-8d0c0ce7-c77b-4f8e-932e-83995be75c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:11:32.938083 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:11:32.996435 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c0ce7-c77b-4f8e-932e-83995be75c37 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:11:33.071796 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5c98d4-162a-4307-ac6c-0163bdbc5de4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:11:33.072569 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c98d4-162a-4307-ac6c-0163bdbc5de4 service cinder] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:11:33.072727 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c98d4-162a-4307-ac6c-0163bdbc5de4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:11:33.072987 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c98d4-162a-4307-ac6c-0163bdbc5de4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:11:33.074661 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-7b5c98d4-162a-4307-ac6c-0163bdbc5de4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:11:33.179224 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5c98d4-162a-4307-ac6c-0163bdbc5de4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:11:33.180676 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5c98d4-162a-4307-ac6c-0163bdbc5de4 service cinder] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:11:33.184321 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:11:33.200790 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 750/1505] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:11:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:11:33.252429 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bbe3a5-4ee2-463d-a0e5-d9d16fee0f82 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:11:33.252869 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bbe3a5-4ee2-463d-a0e5-d9d16fee0f82 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:11:33.252990 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bbe3a5-4ee2-463d-a0e5-d9d16fee0f82 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:11:33.255119 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 756/1506] 10.222.0.60 () {52 vars in 829 bytes} [Wed Jul 13 23:11:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:11:33.259031 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5c98d4-162a-4307-ac6c-0163bdbc5de4 service cinder] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:11:33.260307 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-7b5c98d4-162a-4307-ac6c-0163bdbc5de4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:11:33.362760 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5c98d4-162a-4307-ac6c-0163bdbc5de4 service cinder] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:11:33.462932 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66500bc5-58c5-43a8-bcdb-9150e8e3a890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:11:33.463763 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66500bc5-58c5-43a8-bcdb-9150e8e3a890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:11:33.464047 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66500bc5-58c5-43a8-bcdb-9150e8e3a890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:11:33.464324 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66500bc5-58c5-43a8-bcdb-9150e8e3a890 admin admin] PATH_INFO: `/v3/users/64169e27988c4c4d8449c48427b8419b` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:11:33.478009 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66500bc5-58c5-43a8-bcdb-9150e8e3a890 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64169e27988c4c4d8449c48427b8419b)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:11:33.479072 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: WARNING py.warnings [None req-66500bc5-58c5-43a8-bcdb-9150e8e3a890 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 23:11:33.479072 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: warnings.warn(msg) Jul 13 23:11:33.479072 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: Jul 13 23:11:33.480352 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66500bc5-58c5-43a8-bcdb-9150e8e3a890 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:11:33.490885 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:11:33.556615 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-66500bc5-58c5-43a8-bcdb-9150e8e3a890 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64169e27988c4c4d8449c48427b8419b'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:11:33.557057 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-66500bc5-58c5-43a8-bcdb-9150e8e3a890 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64169e27988c4c4d8449c48427b8419b'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:11:33.575892 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-66500bc5-58c5-43a8-bcdb-9150e8e3a890 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64169e27988c4c4d8449c48427b8419b'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:11:33.588495 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-66500bc5-58c5-43a8-bcdb-9150e8e3a890 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64169e27988c4c4d8449c48427b8419b'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:11:33.589095 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-66500bc5-58c5-43a8-bcdb-9150e8e3a890 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64169e27988c4c4d8449c48427b8419b'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:11:33.605059 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 751/1507] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:11:33 2022] DELETE /identity/v3/users/64169e27988c4c4d8449c48427b8419b => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:11:33.609860 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da204af-6de9-41a1-bdc3-d9b76727ee8d None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:11:33.610167 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da204af-6de9-41a1-bdc3-d9b76727ee8d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:11:33.610445 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da204af-6de9-41a1-bdc3-d9b76727ee8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:11:33.668847 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-7da204af-6de9-41a1-bdc3-d9b76727ee8d None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:11:33.670115 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7da204af-6de9-41a1-bdc3-d9b76727ee8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:11:33.746712 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:11:33.759726 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 757/1508] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:11:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:11:33.769242 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66500bc5-58c5-43a8-bcdb-9150e8e3a890 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:11:33.770455 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-66500bc5-58c5-43a8-bcdb-9150e8e3a890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:11:33.991762 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66500bc5-58c5-43a8-bcdb-9150e8e3a890 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:11:34.046458 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:11:34.069418 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7177c2f-2305-43ab-9fc8-306ebe8134fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:11:34.070078 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7177c2f-2305-43ab-9fc8-306ebe8134fc service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:11:34.070377 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7177c2f-2305-43ab-9fc8-306ebe8134fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:11:34.070653 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7177c2f-2305-43ab-9fc8-306ebe8134fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:11:34.072417 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-b7177c2f-2305-43ab-9fc8-306ebe8134fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:11:34.182808 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7177c2f-2305-43ab-9fc8-306ebe8134fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:11:34.184287 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7177c2f-2305-43ab-9fc8-306ebe8134fc service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:11:34.202404 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 752/1509] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:11:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:11:34.546929 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da204af-6de9-41a1-bdc3-d9b76727ee8d None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:11:34.548470 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7da204af-6de9-41a1-bdc3-d9b76727ee8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:11:34.560585 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da204af-6de9-41a1-bdc3-d9b76727ee8d None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:11:34.656208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93ef55f-1d17-4f67-8488-39efa514d316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:11:34.657093 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ef55f-1d17-4f67-8488-39efa514d316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:11:34.657351 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ef55f-1d17-4f67-8488-39efa514d316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:11:34.657617 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ef55f-1d17-4f67-8488-39efa514d316 admin admin] PATH_INFO: `/v3/projects/8b0032952cf647a985c7b085e0afff08` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:11:34.670604 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93ef55f-1d17-4f67-8488-39efa514d316 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b0032952cf647a985c7b085e0afff08)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:11:34.672180 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: WARNING py.warnings [None req-a93ef55f-1d17-4f67-8488-39efa514d316 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 23:11:34.672180 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: warnings.warn(msg) Jul 13 23:11:34.672180 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: Jul 13 23:11:34.673636 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93ef55f-1d17-4f67-8488-39efa514d316 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:11:34.733555 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a93ef55f-1d17-4f67-8488-39efa514d316 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b0032952cf647a985c7b085e0afff08', 'request_id': 'req-a93ef55f-1d17-4f67-8488-39efa514d316'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:11:34.746968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a93ef55f-1d17-4f67-8488-39efa514d316 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b0032952cf647a985c7b085e0afff08', 'request_id': 'req-a93ef55f-1d17-4f67-8488-39efa514d316'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:11:34.756655 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a93ef55f-1d17-4f67-8488-39efa514d316 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b0032952cf647a985c7b085e0afff08', 'request_id': 'req-a93ef55f-1d17-4f67-8488-39efa514d316'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:11:34.779240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a93ef55f-1d17-4f67-8488-39efa514d316 admin admin] The token cache is being invalidate because project 8b0032952cf647a985c7b085e0afff08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:11:34.779526 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a93ef55f-1d17-4f67-8488-39efa514d316 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:11:34.779985 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a93ef55f-1d17-4f67-8488-39efa514d316 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:11:34.781438 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 758/1510] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:11:34 2022] DELETE /identity/v3/projects/8b0032952cf647a985c7b085e0afff08 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:11:44.200076 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=62 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:11:44.777293 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:13:13.478186 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc331c78-6232-4216-8d1e-72895c9b9133 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:13:13.478826 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc331c78-6232-4216-8d1e-72895c9b9133 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:13:13.478826 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc331c78-6232-4216-8d1e-72895c9b9133 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:13:13.480234 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 753/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 23:13:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:13:13.485719 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e1937-eb06-4cc5-a8fb-8769f1f1ecb7 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:13:13.486261 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e1937-eb06-4cc5-a8fb-8769f1f1ecb7 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:13:13.486261 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e1937-eb06-4cc5-a8fb-8769f1f1ecb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:13:13.576062 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c47e1937-eb06-4cc5-a8fb-8769f1f1ecb7 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:13:13.577313 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c47e1937-eb06-4cc5-a8fb-8769f1f1ecb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:13:13.689470 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 759/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 23:13:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:13:13.844382 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc331c78-6232-4216-8d1e-72895c9b9133 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:13:13.846499 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-cc331c78-6232-4216-8d1e-72895c9b9133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:13:13.967620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc331c78-6232-4216-8d1e-72895c9b9133 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:13:13.973083 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9a62f-2a07-4eb6-bd8b-0e6bfd20ec20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:13:13.973737 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9a62f-2a07-4eb6-bd8b-0e6bfd20ec20 service nova] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:13:13.974041 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9a62f-2a07-4eb6-bd8b-0e6bfd20ec20 service nova] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:13:13.974297 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9a62f-2a07-4eb6-bd8b-0e6bfd20ec20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:13:13.982852 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f9a62f-2a07-4eb6-bd8b-0e6bfd20ec20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:13:13.984507 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f9a62f-2a07-4eb6-bd8b-0e6bfd20ec20 service nova] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:13:13.994865 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 754/1513] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:13:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:13:14.840130 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520123f8-ed15-483f-883c-a9c72bc4a4f7 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:13:14.840668 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520123f8-ed15-483f-883c-a9c72bc4a4f7 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:13:14.840668 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520123f8-ed15-483f-883c-a9c72bc4a4f7 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:13:14.842063 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 760/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 23:13:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:13:14.847724 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84110e9a-bf3d-41c1-ba0b-a2e03ea25554 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:13:14.848175 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84110e9a-bf3d-41c1-ba0b-a2e03ea25554 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:13:14.848301 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84110e9a-bf3d-41c1-ba0b-a2e03ea25554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:13:14.866938 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-84110e9a-bf3d-41c1-ba0b-a2e03ea25554 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:13:14.868178 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-84110e9a-bf3d-41c1-ba0b-a2e03ea25554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:13:14.876099 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 755/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 23:13:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:13:15.027028 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520123f8-ed15-483f-883c-a9c72bc4a4f7 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:13:15.041904 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520123f8-ed15-483f-883c-a9c72bc4a4f7 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:13:15.047321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de12804f-c0c2-4261-894f-e720b850c2c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:13:15.047988 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de12804f-c0c2-4261-894f-e720b850c2c2 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:13:15.048267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de12804f-c0c2-4261-894f-e720b850c2c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:13:15.048541 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de12804f-c0c2-4261-894f-e720b850c2c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:13:15.056772 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de12804f-c0c2-4261-894f-e720b850c2c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:13:15.058356 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de12804f-c0c2-4261-894f-e720b850c2c2 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:13:15.068609 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 761/1516] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:13:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:13:15.999872 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e359ece6-742f-4fb1-8f12-8317f234daf0 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:13:16.000457 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e359ece6-742f-4fb1-8f12-8317f234daf0 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:13:16.000457 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e359ece6-742f-4fb1-8f12-8317f234daf0 None None] PATH_INFO: `/v3` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:13:16.001738 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 756/1517] 10.222.0.58 () {54 vars in 925 bytes} [Wed Jul 13 23:13:15 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:13:16.007939 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1ce86-89cb-4bc8-8df9-3831983fefee None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:13:16.008409 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1ce86-89cb-4bc8-8df9-3831983fefee None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:13:16.008516 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1ce86-89cb-4bc8-8df9-3831983fefee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:13:16.066793 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-dca1ce86-89cb-4bc8-8df9-3831983fefee None None] MFA Rules not processed for user `64824ba84a024930bbc177f8a8d9a046`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:13:16.068034 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-dca1ce86-89cb-4bc8-8df9-3831983fefee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:13:16.084493 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:13:16.140968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 762/1518] 10.222.0.58 () {58 vars in 1015 bytes} [Wed Jul 13 23:13:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:13:20.018687 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abaa092-e6ac-4364-b452-cf68d3b8af14 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:13:20.019333 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abaa092-e6ac-4364-b452-cf68d3b8af14 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:13:20.019333 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abaa092-e6ac-4364-b452-cf68d3b8af14 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:13:20.020580 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 757/1519] 10.222.0.60 () {54 vars in 960 bytes} [Wed Jul 13 23:13:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:13:20.025447 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475144a-a4d9-4bb6-84ae-6f608d4c717d None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:13:20.025909 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475144a-a4d9-4bb6-84ae-6f608d4c717d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:13:20.026011 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475144a-a4d9-4bb6-84ae-6f608d4c717d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:13:20.044536 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-9475144a-a4d9-4bb6-84ae-6f608d4c717d None None] MFA Rules not processed for user `64824ba84a024930bbc177f8a8d9a046`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:13:20.045744 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9475144a-a4d9-4bb6-84ae-6f608d4c717d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:13:20.054310 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 763/1520] 10.222.0.60 () {58 vars in 1057 bytes} [Wed Jul 13 23:13:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:13:20.062195 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abaa092-e6ac-4364-b452-cf68d3b8af14 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:13:20.075851 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abaa092-e6ac-4364-b452-cf68d3b8af14 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:13:20.081433 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614c2e78-85e0-4eed-b794-5d89f65871f0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64824ba84a024930bbc177f8a8d9a046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:13:20.082083 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c2e78-85e0-4eed-b794-5d89f65871f0 service placement] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:13:20.082375 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c2e78-85e0-4eed-b794-5d89f65871f0 service placement] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:13:20.082682 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c2e78-85e0-4eed-b794-5d89f65871f0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:13:20.091327 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614c2e78-85e0-4eed-b794-5d89f65871f0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:13:20.092969 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614c2e78-85e0-4eed-b794-5d89f65871f0 service placement] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:13:20.103327 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 758/1521] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:13:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:13:26.297622 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8736c6d3-84b0-4178-ae4f-2c2761ee1abc None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:13:26.298205 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8736c6d3-84b0-4178-ae4f-2c2761ee1abc None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:13:26.298205 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8736c6d3-84b0-4178-ae4f-2c2761ee1abc None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:13:26.299650 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 764/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 23:13:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:13:26.304505 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b56325-c854-409d-95fe-9921a6f4f036 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:13:26.304938 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b56325-c854-409d-95fe-9921a6f4f036 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:13:26.305043 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b56325-c854-409d-95fe-9921a6f4f036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:13:26.323630 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-96b56325-c854-409d-95fe-9921a6f4f036 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:13:26.324842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-96b56325-c854-409d-95fe-9921a6f4f036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:13:26.333444 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 759/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 23:13:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:13:26.485572 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8736c6d3-84b0-4178-ae4f-2c2761ee1abc None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:13:26.500059 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8736c6d3-84b0-4178-ae4f-2c2761ee1abc None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:13:26.505997 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5eb26d-a01c-423a-bc68-06c89cca981e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:13:26.506625 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5eb26d-a01c-423a-bc68-06c89cca981e service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:13:26.506908 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5eb26d-a01c-423a-bc68-06c89cca981e service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:13:26.507183 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5eb26d-a01c-423a-bc68-06c89cca981e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:13:26.515237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5eb26d-a01c-423a-bc68-06c89cca981e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:13:26.516851 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5eb26d-a01c-423a-bc68-06c89cca981e service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:13:26.527850 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 765/1524] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:13:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:13:27.350274 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26b7ba-597c-4eca-bd36-c3ac2d0e96e3 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:13:27.350875 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26b7ba-597c-4eca-bd36-c3ac2d0e96e3 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:13:27.350875 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26b7ba-597c-4eca-bd36-c3ac2d0e96e3 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:13:27.352244 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 760/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 23:13:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:13:27.357067 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53e7b3-6640-46d9-95a8-04a5838a251c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:13:27.357503 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53e7b3-6640-46d9-95a8-04a5838a251c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:13:27.357662 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53e7b3-6640-46d9-95a8-04a5838a251c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:13:27.376149 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-3e53e7b3-6640-46d9-95a8-04a5838a251c None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:13:27.377365 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3e53e7b3-6640-46d9-95a8-04a5838a251c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:13:27.385462 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 766/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 23:13:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:13:27.539356 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e26b7ba-597c-4eca-bd36-c3ac2d0e96e3 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:13:27.554694 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e26b7ba-597c-4eca-bd36-c3ac2d0e96e3 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:13:27.560466 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd87334c-977f-436d-be3d-96f8a5baba4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:13:27.561102 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87334c-977f-436d-be3d-96f8a5baba4c service nova] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:13:27.561379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87334c-977f-436d-be3d-96f8a5baba4c service nova] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:13:27.561664 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87334c-977f-436d-be3d-96f8a5baba4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:13:27.570113 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd87334c-977f-436d-be3d-96f8a5baba4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:13:27.571757 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd87334c-977f-436d-be3d-96f8a5baba4c service nova] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:13:27.581887 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 761/1527] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:13:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:13:31.534691 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4454a-43a1-43cd-bb83-11c6238a9310 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:13:31.535287 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4454a-43a1-43cd-bb83-11c6238a9310 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:13:31.535287 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4454a-43a1-43cd-bb83-11c6238a9310 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:13:31.536663 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 767/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 23:13:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:13:31.541762 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6071c40-f3a0-4989-b0cc-943ee644d591 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:13:31.542183 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6071c40-f3a0-4989-b0cc-943ee644d591 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:13:31.542318 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6071c40-f3a0-4989-b0cc-943ee644d591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:13:31.561881 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-f6071c40-f3a0-4989-b0cc-943ee644d591 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:13:31.563085 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-f6071c40-f3a0-4989-b0cc-943ee644d591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:13:31.570966 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 762/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 23:13:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:13:31.644206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b4454a-43a1-43cd-bb83-11c6238a9310 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:13:31.645917 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-51b4454a-43a1-43cd-bb83-11c6238a9310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:13:31.724199 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b4454a-43a1-43cd-bb83-11c6238a9310 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:13:31.727980 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4c5c1a-88c0-4a69-98d2-a926b2abf832 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:13:31.728620 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c5c1a-88c0-4a69-98d2-a926b2abf832 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:13:31.728903 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c5c1a-88c0-4a69-98d2-a926b2abf832 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:13:31.729178 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c5c1a-88c0-4a69-98d2-a926b2abf832 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:13:31.739715 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4c5c1a-88c0-4a69-98d2-a926b2abf832 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:13:31.741141 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4c5c1a-88c0-4a69-98d2-a926b2abf832 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:13:31.750437 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 768/1530] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:13:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:13:33.761501 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2372c6-cb33-4bd2-85a6-55534ea97f64 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:13:33.762046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2372c6-cb33-4bd2-85a6-55534ea97f64 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:13:33.762046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2372c6-cb33-4bd2-85a6-55534ea97f64 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:13:33.763407 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 763/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 23:13:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:13:33.769197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848fa07-eb5a-4010-8f1f-d9d52de103c3 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:13:33.769636 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848fa07-eb5a-4010-8f1f-d9d52de103c3 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:13:33.769759 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848fa07-eb5a-4010-8f1f-d9d52de103c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:13:33.788023 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-8848fa07-eb5a-4010-8f1f-d9d52de103c3 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:13:33.789220 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8848fa07-eb5a-4010-8f1f-d9d52de103c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:13:33.797137 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 769/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 23:13:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:13:33.834047 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2372c6-cb33-4bd2-85a6-55534ea97f64 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:13:33.844620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2372c6-cb33-4bd2-85a6-55534ea97f64 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:13:33.848049 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8059e-1008-4aec-bb68-bd856f925d67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:13:33.848671 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8059e-1008-4aec-bb68-bd856f925d67 service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:13:33.848950 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8059e-1008-4aec-bb68-bd856f925d67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:13:33.849217 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8059e-1008-4aec-bb68-bd856f925d67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:13:33.857879 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c8059e-1008-4aec-bb68-bd856f925d67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:13:33.859298 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c8059e-1008-4aec-bb68-bd856f925d67 service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:13:33.868501 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 764/1533] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:13:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:13:37.472721 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282b78e-3222-4c13-8956-aa962aee9fb8 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:13:37.473404 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282b78e-3222-4c13-8956-aa962aee9fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:13:37.473404 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282b78e-3222-4c13-8956-aa962aee9fb8 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:13:37.474669 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 770/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 23:13:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:13:37.479848 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99857f-788a-4b49-bbd2-b43e0da0960f None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:13:37.480293 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99857f-788a-4b49-bbd2-b43e0da0960f None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:13:37.480413 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99857f-788a-4b49-bbd2-b43e0da0960f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:13:37.487110 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=81 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:13:37.499727 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-5d99857f-788a-4b49-bbd2-b43e0da0960f None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:13:37.500941 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-5d99857f-788a-4b49-bbd2-b43e0da0960f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:13:37.509339 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 765/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 23:13:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:13:37.813906 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5282b78e-3222-4c13-8956-aa962aee9fb8 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:13:37.815620 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5282b78e-3222-4c13-8956-aa962aee9fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:13:37.879747 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:13:37.892902 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5282b78e-3222-4c13-8956-aa962aee9fb8 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:13:37.896533 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d057a3c-f6d1-4a6b-8e34-5e549ba3b3b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:13:37.897176 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d057a3c-f6d1-4a6b-8e34-5e549ba3b3b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:13:37.897459 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d057a3c-f6d1-4a6b-8e34-5e549ba3b3b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:13:37.897731 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d057a3c-f6d1-4a6b-8e34-5e549ba3b3b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:13:37.906911 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d057a3c-f6d1-4a6b-8e34-5e549ba3b3b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:13:37.908877 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d057a3c-f6d1-4a6b-8e34-5e549ba3b3b1 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:13:37.918329 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 771/1536] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:13:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:13:39.970767 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc1f0e-da94-4094-af3c-8acd687c05ca None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:13:39.971350 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc1f0e-da94-4094-af3c-8acd687c05ca None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:13:39.971350 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc1f0e-da94-4094-af3c-8acd687c05ca None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:13:39.972724 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 766/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 23:13:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:13:39.977810 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a36ae3-93b4-4c10-806c-49984c47a451 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:13:39.978242 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a36ae3-93b4-4c10-806c-49984c47a451 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:13:39.978377 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a36ae3-93b4-4c10-806c-49984c47a451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:13:39.995764 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-64a36ae3-93b4-4c10-806c-49984c47a451 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:13:39.996997 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-64a36ae3-93b4-4c10-806c-49984c47a451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:13:40.005406 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 772/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 23:13:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:13:40.308130 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc1f0e-da94-4094-af3c-8acd687c05ca None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:13:40.310048 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-a3fc1f0e-da94-4094-af3c-8acd687c05ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:13:40.323429 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc1f0e-da94-4094-af3c-8acd687c05ca None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:13:40.326906 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddccfc0-7854-41b3-9ff8-b77be10a35fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:13:40.327569 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddccfc0-7854-41b3-9ff8-b77be10a35fa service cinder] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:13:40.327845 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddccfc0-7854-41b3-9ff8-b77be10a35fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:13:40.328121 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddccfc0-7854-41b3-9ff8-b77be10a35fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:13:40.336981 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddccfc0-7854-41b3-9ff8-b77be10a35fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:13:40.338418 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddccfc0-7854-41b3-9ff8-b77be10a35fa service cinder] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:13:40.350970 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 767/1539] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:13:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:13:49.990893 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=11 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:13:50.345949 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=8 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:09.260919 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ecbb36-8b9d-43ce-b37f-adc51e7843ed None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:09.262217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ecbb36-8b9d-43ce-b37f-adc51e7843ed None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:09.262720 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ecbb36-8b9d-43ce-b37f-adc51e7843ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:09.297323 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-66ecbb36-8b9d-43ce-b37f-adc51e7843ed None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:09.299091 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-66ecbb36-8b9d-43ce-b37f-adc51e7843ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:09.312467 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 773/1540] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:15:09.322142 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ad8b3-113a-4442-85c3-b6b349f961a7 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:09.322745 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ad8b3-113a-4442-85c3-b6b349f961a7 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:09.322745 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ad8b3-113a-4442-85c3-b6b349f961a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:09.350872 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c287f-6fe7-4038-98cb-378b7995b970 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:09.351525 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c287f-6fe7-4038-98cb-378b7995b970 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:09.352164 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-847ad8b3-113a-4442-85c3-b6b349f961a7 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:09.353064 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c287f-6fe7-4038-98cb-378b7995b970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:09.353539 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-847ad8b3-113a-4442-85c3-b6b349f961a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:09.366409 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 768/1541] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:09.367761 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205fd8e-be5b-4314-9554-7aff8d304f3d None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:09.368050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205fd8e-be5b-4314-9554-7aff8d304f3d None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:09.368329 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205fd8e-be5b-4314-9554-7aff8d304f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:09.393494 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-ab7c287f-6fe7-4038-98cb-378b7995b970 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:09.394875 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ab7c287f-6fe7-4038-98cb-378b7995b970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:09.409339 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 774/1542] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:09.410412 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c287f-6fe7-4038-98cb-378b7995b970 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:09.417251 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-5205fd8e-be5b-4314-9554-7aff8d304f3d None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:09.418873 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-5205fd8e-be5b-4314-9554-7aff8d304f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:09.427619 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c287f-6fe7-4038-98cb-378b7995b970 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:09.439453 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31b0901-a034-4046-8d6e-69085577c79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:09.440179 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b0901-a034-4046-8d6e-69085577c79b admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:09.440463 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b0901-a034-4046-8d6e-69085577c79b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:09.440735 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b0901-a034-4046-8d6e-69085577c79b admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:09.441512 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31b0901-a034-4046-8d6e-69085577c79b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:15:09.441820 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31b0901-a034-4046-8d6e-69085577c79b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:09.443504 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31b0901-a034-4046-8d6e-69085577c79b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:09.445187 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 769/1543] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:09.446582 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdad9dc-6262-4aeb-bdca-e1b831d6907f None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:09.446874 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdad9dc-6262-4aeb-bdca-e1b831d6907f None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:09.447563 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdad9dc-6262-4aeb-bdca-e1b831d6907f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:09.467637 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 775/1544] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:15:09.470250 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bdee24-74f3-4254-9a5d-d0cacb229def None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:09.470542 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bdee24-74f3-4254-9a5d-d0cacb229def None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:09.470818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bdee24-74f3-4254-9a5d-d0cacb229def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:09.497642 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-2bdad9dc-6262-4aeb-bdca-e1b831d6907f None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:09.498462 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-2bdad9dc-6262-4aeb-bdca-e1b831d6907f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:09.520345 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-26bdee24-74f3-4254-9a5d-d0cacb229def None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:09.522005 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-26bdee24-74f3-4254-9a5d-d0cacb229def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:09.522652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 770/1545] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:09.523497 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdad9dc-6262-4aeb-bdca-e1b831d6907f None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:09.543364 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 776/1546] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:09.544053 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bdee24-74f3-4254-9a5d-d0cacb229def None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:09.550747 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdad9dc-6262-4aeb-bdca-e1b831d6907f None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:09.559381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bdee24-74f3-4254-9a5d-d0cacb229def None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:09.565944 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40114199-f86e-4910-b00e-1e207013ed91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:09.566642 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40114199-f86e-4910-b00e-1e207013ed91 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:09.566927 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40114199-f86e-4910-b00e-1e207013ed91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:09.567204 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40114199-f86e-4910-b00e-1e207013ed91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:09.567973 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40114199-f86e-4910-b00e-1e207013ed91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:09.570369 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40114199-f86e-4910-b00e-1e207013ed91 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:09.573123 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6104ab-deb2-4df6-ae6e-c037a861928a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:09.573819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6104ab-deb2-4df6-ae6e-c037a861928a admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:09.574101 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6104ab-deb2-4df6-ae6e-c037a861928a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:09.574376 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6104ab-deb2-4df6-ae6e-c037a861928a admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:09.575113 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6104ab-deb2-4df6-ae6e-c037a861928a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:15:09.575417 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6104ab-deb2-4df6-ae6e-c037a861928a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:09.577070 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6104ab-deb2-4df6-ae6e-c037a861928a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:09.602113 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 777/1547] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:15:09.603258 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6104ab-deb2-4df6-ae6e-c037a861928a admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:09.621617 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6104ab-deb2-4df6-ae6e-c037a861928a admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:09.632226 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e8a5d5-a852-4e4a-87bf-b75c67682c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:09.632903 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8a5d5-a852-4e4a-87bf-b75c67682c97 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:09.633277 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8a5d5-a852-4e4a-87bf-b75c67682c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:09.633498 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8a5d5-a852-4e4a-87bf-b75c67682c97 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:09.634355 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e8a5d5-a852-4e4a-87bf-b75c67682c97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:15:09.634651 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e8a5d5-a852-4e4a-87bf-b75c67682c97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:09.636885 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e8a5d5-a852-4e4a-87bf-b75c67682c97 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:09.666076 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 771/1548] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:15:09 2022] POST /identity/v3/projects => generated 379 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:09.666283 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 778/1549] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:15:09.666566 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40114199-f86e-4910-b00e-1e207013ed91 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:09.672602 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e8a5d5-a852-4e4a-87bf-b75c67682c97 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:09.695103 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e8a5d5-a852-4e4a-87bf-b75c67682c97 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:09.809708 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40114199-f86e-4910-b00e-1e207013ed91 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:09.821648 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a6230b-5ff8-47a6-8c94-76765c94a82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:09.822330 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6230b-5ff8-47a6-8c94-76765c94a82d admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:09.822620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6230b-5ff8-47a6-8c94-76765c94a82d admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:09.822898 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6230b-5ff8-47a6-8c94-76765c94a82d admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:09.824853 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a6230b-5ff8-47a6-8c94-76765c94a82d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:09.826861 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a6230b-5ff8-47a6-8c94-76765c94a82d admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:09.829874 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993036de-5d30-47d4-9319-ab199896b11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:09.830569 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993036de-5d30-47d4-9319-ab199896b11d admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:09.830850 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993036de-5d30-47d4-9319-ab199896b11d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:09.831127 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993036de-5d30-47d4-9319-ab199896b11d admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:09.831862 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993036de-5d30-47d4-9319-ab199896b11d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:09.834841 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993036de-5d30-47d4-9319-ab199896b11d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:09.860387 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 779/1550] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:15:09 2022] POST /identity/v3/users => generated 290 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:09.861296 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993036de-5d30-47d4-9319-ab199896b11d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:09.871878 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 772/1551] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:15:09 2022] POST /identity/v3/projects => generated 405 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:09.873228 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a491979-c60d-42fa-af65-b450ed21ca68 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:09.873624 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a491979-c60d-42fa-af65-b450ed21ca68 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:09.873922 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993036de-5d30-47d4-9319-ab199896b11d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:09.874065 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a491979-c60d-42fa-af65-b450ed21ca68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:09.899786 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-9a491979-c60d-42fa-af65-b450ed21ca68 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:09.901327 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-9a491979-c60d-42fa-af65-b450ed21ca68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:09.941539 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ece09-148e-4473-a666-c8c357fea3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:09.942209 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ece09-148e-4473-a666-c8c357fea3ed admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:09.942494 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ece09-148e-4473-a666-c8c357fea3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:09.942771 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ece09-148e-4473-a666-c8c357fea3ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:09.943474 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2ece09-148e-4473-a666-c8c357fea3ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:09.944536 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: WARNING py.warnings [None req-ee2ece09-148e-4473-a666-c8c357fea3ed admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 23:15:09.944536 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: warnings.warn(msg) Jul 13 23:15:09.944536 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: Jul 13 23:15:09.945908 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2ece09-148e-4473-a666-c8c357fea3ed admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:09.966464 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 773/1552] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:09.967763 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44609c91-1ca5-4d98-8ebc-f7aca00b3be9 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:09.968088 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44609c91-1ca5-4d98-8ebc-f7aca00b3be9 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:09.968364 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44609c91-1ca5-4d98-8ebc-f7aca00b3be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:09.974403 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 780/1553] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:15:09 2022] POST /identity/v3/projects => generated 389 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:09.975799 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fadbc4-6e56-4d14-b17c-71c36d8187fa None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:09.976086 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fadbc4-6e56-4d14-b17c-71c36d8187fa None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:09.976361 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fadbc4-6e56-4d14-b17c-71c36d8187fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:09.989517 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-44609c91-1ca5-4d98-8ebc-f7aca00b3be9 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:09.990756 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-44609c91-1ca5-4d98-8ebc-f7aca00b3be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:09.993991 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-e4fadbc4-6e56-4d14-b17c-71c36d8187fa None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:09.995504 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e4fadbc4-6e56-4d14-b17c-71c36d8187fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:10.005284 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:10.022814 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:10.053049 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 774/1554] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:10.053973 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44609c91-1ca5-4d98-8ebc-f7aca00b3be9 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:10.064360 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44609c91-1ca5-4d98-8ebc-f7aca00b3be9 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:10.065688 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 781/1555] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:10.066510 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fadbc4-6e56-4d14-b17c-71c36d8187fa None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:10.068892 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7f97b-8fa8-43d3-a833-011b9aaf7099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:10.069557 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7f97b-8fa8-43d3-a833-011b9aaf7099 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:10.069836 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7f97b-8fa8-43d3-a833-011b9aaf7099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:10.070105 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7f97b-8fa8-43d3-a833-011b9aaf7099 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:10.070826 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e7f97b-8fa8-43d3-a833-011b9aaf7099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:10.072564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e7f97b-8fa8-43d3-a833-011b9aaf7099 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:10.079341 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fadbc4-6e56-4d14-b17c-71c36d8187fa None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:10.083578 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 775/1556] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:10.085066 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0aa3e-be8b-4a64-befa-709e3151dfc2 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:10.085359 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0aa3e-be8b-4a64-befa-709e3151dfc2 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:10.085634 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0aa3e-be8b-4a64-befa-709e3151dfc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:10.086169 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a210af5-b1c3-4be0-9da1-aad86aae2f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:10.087132 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a210af5-b1c3-4be0-9da1-aad86aae2f93 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:10.087424 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a210af5-b1c3-4be0-9da1-aad86aae2f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:10.087737 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a210af5-b1c3-4be0-9da1-aad86aae2f93 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:10.088428 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a210af5-b1c3-4be0-9da1-aad86aae2f93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:10.090360 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a210af5-b1c3-4be0-9da1-aad86aae2f93 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:10.104829 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-fcf0aa3e-be8b-4a64-befa-709e3151dfc2 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:10.106328 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-fcf0aa3e-be8b-4a64-befa-709e3151dfc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:10.111424 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 782/1557] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:15:10 2022] POST /identity/v3/users => generated 303 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:10.112741 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6bf8e9-db78-4687-b06e-ad5bbbbb01f4 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:10.113031 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6bf8e9-db78-4687-b06e-ad5bbbbb01f4 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:10.113309 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6bf8e9-db78-4687-b06e-ad5bbbbb01f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:10.115435 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 776/1558] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:10.124600 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b11ef0-8e31-4693-a498-f6f2760ac8c9 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:10.124888 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b11ef0-8e31-4693-a498-f6f2760ac8c9 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:10.125318 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b11ef0-8e31-4693-a498-f6f2760ac8c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:10.132309 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-bd6bf8e9-db78-4687-b06e-ad5bbbbb01f4 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:10.133433 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-bd6bf8e9-db78-4687-b06e-ad5bbbbb01f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:10.142537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 783/1559] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:10.143329 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6bf8e9-db78-4687-b06e-ad5bbbbb01f4 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:10.151187 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-52b11ef0-8e31-4693-a498-f6f2760ac8c9 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:10.152506 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-52b11ef0-8e31-4693-a498-f6f2760ac8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:10.153106 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6bf8e9-db78-4687-b06e-ad5bbbbb01f4 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:10.157743 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf3a8e-55d0-4572-9573-166b52bf836a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:10.158403 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf3a8e-55d0-4572-9573-166b52bf836a admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:10.158687 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf3a8e-55d0-4572-9573-166b52bf836a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:10.158961 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf3a8e-55d0-4572-9573-166b52bf836a admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:10.159713 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf3a8e-55d0-4572-9573-166b52bf836a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:10.160763 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 777/1560] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:10.161638 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf3a8e-55d0-4572-9573-166b52bf836a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:10.162007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0394b-fe61-4802-af3c-ad43d8fd6c40 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:10.162284 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0394b-fe61-4802-af3c-ad43d8fd6c40 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:10.162561 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0394b-fe61-4802-af3c-ad43d8fd6c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:10.182379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-9ba0394b-fe61-4802-af3c-ad43d8fd6c40 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:10.183643 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-9ba0394b-fe61-4802-af3c-ad43d8fd6c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:10.185003 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 784/1561] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:15:10 2022] POST /identity/v3/users => generated 295 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:10.186284 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2d5a1c-4c94-4991-a199-5d387cca2f61 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:10.186603 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2d5a1c-4c94-4991-a199-5d387cca2f61 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:10.187399 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2d5a1c-4c94-4991-a199-5d387cca2f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:10.195175 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 778/1562] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:10.195175 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba0394b-fe61-4802-af3c-ad43d8fd6c40 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:10.206558 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-2a2d5a1c-4c94-4991-a199-5d387cca2f61 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:10.207798 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-2a2d5a1c-4c94-4991-a199-5d387cca2f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:10.208238 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba0394b-fe61-4802-af3c-ad43d8fd6c40 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:10.212903 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b0311-3f6e-4a8f-a4a8-2fd31a77bb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:10.213562 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b0311-3f6e-4a8f-a4a8-2fd31a77bb2b admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:10.213843 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b0311-3f6e-4a8f-a4a8-2fd31a77bb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:10.214116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b0311-3f6e-4a8f-a4a8-2fd31a77bb2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:10.214996 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69b0311-3f6e-4a8f-a4a8-2fd31a77bb2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:10.215945 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 785/1563] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:10.216645 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69b0311-3f6e-4a8f-a4a8-2fd31a77bb2b admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:10.216771 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2d5a1c-4c94-4991-a199-5d387cca2f61 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:10.226050 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2d5a1c-4c94-4991-a199-5d387cca2f61 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:10.227237 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 779/1564] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:10.228528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76321b-285e-484b-9652-fc64c56e659a None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:10.228816 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76321b-285e-484b-9652-fc64c56e659a None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:10.229090 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76321b-285e-484b-9652-fc64c56e659a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:10.234249 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f3a94e-8455-43ab-ab94-41f33d7f7fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:10.234919 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3a94e-8455-43ab-ab94-41f33d7f7fdc admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:10.235193 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3a94e-8455-43ab-ab94-41f33d7f7fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:10.235462 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3a94e-8455-43ab-ab94-41f33d7f7fdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:10.236218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f3a94e-8455-43ab-ab94-41f33d7f7fdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:10.237819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f3a94e-8455-43ab-ab94-41f33d7f7fdc admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:10.247406 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-1f76321b-285e-484b-9652-fc64c56e659a None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:10.248453 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 786/1565] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:10.248646 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-1f76321b-285e-484b-9652-fc64c56e659a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:10.250351 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb3a1f6-9e2c-4bf2-a1d0-af224b8a9e7f None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:10.250636 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb3a1f6-9e2c-4bf2-a1d0-af224b8a9e7f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:10.250904 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb3a1f6-9e2c-4bf2-a1d0-af224b8a9e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:10.258512 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 780/1566] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:10.259374 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76321b-285e-484b-9652-fc64c56e659a None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:10.269669 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76321b-285e-484b-9652-fc64c56e659a None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:10.271093 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-cfb3a1f6-9e2c-4bf2-a1d0-af224b8a9e7f None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:10.272348 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-cfb3a1f6-9e2c-4bf2-a1d0-af224b8a9e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:10.274790 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a38831-6e9a-4fdf-8474-f4dd65e939a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:10.275784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a38831-6e9a-4fdf-8474-f4dd65e939a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:10.276281 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a38831-6e9a-4fdf-8474-f4dd65e939a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:10.276636 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a38831-6e9a-4fdf-8474-f4dd65e939a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:10.277377 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a38831-6e9a-4fdf-8474-f4dd65e939a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:15:10.277686 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a38831-6e9a-4fdf-8474-f4dd65e939a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:10.278736 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: WARNING py.warnings [None req-74a38831-6e9a-4fdf-8474-f4dd65e939a8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 13 23:15:10.278736 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: warnings.warn(msg) Jul 13 23:15:10.278736 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: Jul 13 23:15:10.279642 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a38831-6e9a-4fdf-8474-f4dd65e939a8 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:10.280635 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 787/1567] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:10.281470 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb3a1f6-9e2c-4bf2-a1d0-af224b8a9e7f None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:10.291407 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb3a1f6-9e2c-4bf2-a1d0-af224b8a9e7f None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:10.301541 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 781/1568] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:15:10.301541 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a38831-6e9a-4fdf-8474-f4dd65e939a8 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:10.301870 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63821539-ea74-44cb-a559-b89d0408c846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:10.301870 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63821539-ea74-44cb-a559-b89d0408c846 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:10.301870 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63821539-ea74-44cb-a559-b89d0408c846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:10.301870 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63821539-ea74-44cb-a559-b89d0408c846 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:10.302234 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63821539-ea74-44cb-a559-b89d0408c846 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:10.302234 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63821539-ea74-44cb-a559-b89d0408c846 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:10.306486 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a38831-6e9a-4fdf-8474-f4dd65e939a8 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:10.312387 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a87ab-5b4e-4dfc-a9a2-914e57d59373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:10.313035 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a87ab-5b4e-4dfc-a9a2-914e57d59373 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:10.313165 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 788/1569] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:10.313369 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a87ab-5b4e-4dfc-a9a2-914e57d59373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:10.313586 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a87ab-5b4e-4dfc-a9a2-914e57d59373 admin admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:10.314227 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63821539-ea74-44cb-a559-b89d0408c846 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:10.314353 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728a87ab-5b4e-4dfc-a9a2-914e57d59373 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:15:10.314654 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728a87ab-5b4e-4dfc-a9a2-914e57d59373 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:10.316297 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728a87ab-5b4e-4dfc-a9a2-914e57d59373 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:10.323882 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63821539-ea74-44cb-a559-b89d0408c846 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:10.328907 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612f82a6-0bc0-476e-8d2d-59fbed59c28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:10.329311 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 782/1570] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:15:10.329822 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f82a6-0bc0-476e-8d2d-59fbed59c28b admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:10.330065 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f82a6-0bc0-476e-8d2d-59fbed59c28b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:10.330176 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a87ab-5b4e-4dfc-a9a2-914e57d59373 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:10.330267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f82a6-0bc0-476e-8d2d-59fbed59c28b admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:10.331002 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612f82a6-0bc0-476e-8d2d-59fbed59c28b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:15:10.331343 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612f82a6-0bc0-476e-8d2d-59fbed59c28b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:10.333384 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612f82a6-0bc0-476e-8d2d-59fbed59c28b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:10.340038 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a87ab-5b4e-4dfc-a9a2-914e57d59373 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:10.344676 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472dd23-f751-4e31-aa6f-09f6b1815c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:10.345558 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472dd23-f751-4e31-aa6f-09f6b1815c07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:10.345839 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472dd23-f751-4e31-aa6f-09f6b1815c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:10.346110 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472dd23-f751-4e31-aa6f-09f6b1815c07 admin admin] PATH_INFO: `/v3/projects/5c873a6e9bdd4b74bcea03fb45317291/users/7ade2669af504e91936edb12c5d919a6/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:10.352632 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 789/1571] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:15:10.352632 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612f82a6-0bc0-476e-8d2d-59fbed59c28b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:10.357834 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612f82a6-0bc0-476e-8d2d-59fbed59c28b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:10.365412 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aae826a-f6e6-47ed-a673-d9d7a6c7b5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:10.366113 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae826a-f6e6-47ed-a673-d9d7a6c7b5b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:10.366395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae826a-f6e6-47ed-a673-d9d7a6c7b5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:10.366676 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aae826a-f6e6-47ed-a673-d9d7a6c7b5b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:10.367423 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aae826a-f6e6-47ed-a673-d9d7a6c7b5b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:10.369131 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aae826a-f6e6-47ed-a673-d9d7a6c7b5b9 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:10.369333 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e472dd23-f751-4e31-aa6f-09f6b1815c07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c873a6e9bdd4b74bcea03fb45317291, user_id=7ade2669af504e91936edb12c5d919a6, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:10.372161 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e472dd23-f751-4e31-aa6f-09f6b1815c07 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:10.381462 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 790/1572] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:10.385212 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aae826a-f6e6-47ed-a673-d9d7a6c7b5b9 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:10.391844 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 783/1573] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:15:10 2022] PUT /identity/v3/projects/5c873a6e9bdd4b74bcea03fb45317291/users/7ade2669af504e91936edb12c5d919a6/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:10.393548 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aae826a-f6e6-47ed-a673-d9d7a6c7b5b9 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:10.396706 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472dd23-f751-4e31-aa6f-09f6b1815c07 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:10.454625 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79932fa-dc14-46df-8a3c-02fa3b6625e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:10.455375 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79932fa-dc14-46df-8a3c-02fa3b6625e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:10.455755 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79932fa-dc14-46df-8a3c-02fa3b6625e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:10.456107 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79932fa-dc14-46df-8a3c-02fa3b6625e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:10.456954 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79932fa-dc14-46df-8a3c-02fa3b6625e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:15:10.457319 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79932fa-dc14-46df-8a3c-02fa3b6625e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:10.459044 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79932fa-dc14-46df-8a3c-02fa3b6625e5 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:10.459922 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472dd23-f751-4e31-aa6f-09f6b1815c07 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:10.464612 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef93423-5775-47ea-af57-973681dc4c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:10.465278 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef93423-5775-47ea-af57-973681dc4c79 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:10.465564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef93423-5775-47ea-af57-973681dc4c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:10.465839 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef93423-5775-47ea-af57-973681dc4c79 admin admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:10.466600 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef93423-5775-47ea-af57-973681dc4c79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:15:10.466906 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef93423-5775-47ea-af57-973681dc4c79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:10.468634 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef93423-5775-47ea-af57-973681dc4c79 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:10.473731 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 791/1574] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:15:10.474548 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79932fa-dc14-46df-8a3c-02fa3b6625e5 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:10.482128 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 784/1575] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:15:10.484896 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef93423-5775-47ea-af57-973681dc4c79 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:10.487603 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79932fa-dc14-46df-8a3c-02fa3b6625e5 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:10.492468 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1322aa-80b3-48c8-9dfd-9c28b4c50f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:10.493124 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1322aa-80b3-48c8-9dfd-9c28b4c50f87 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:10.493402 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1322aa-80b3-48c8-9dfd-9c28b4c50f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:10.493675 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1322aa-80b3-48c8-9dfd-9c28b4c50f87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:10.494745 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1322aa-80b3-48c8-9dfd-9c28b4c50f87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:10.495733 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef93423-5775-47ea-af57-973681dc4c79 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:10.496299 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1322aa-80b3-48c8-9dfd-9c28b4c50f87 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:10.501127 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51774b00-6a8d-4197-8e5e-247af8edbb6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:10.501289 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:10.501790 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51774b00-6a8d-4197-8e5e-247af8edbb6d admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:10.502075 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51774b00-6a8d-4197-8e5e-247af8edbb6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:10.502346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51774b00-6a8d-4197-8e5e-247af8edbb6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:10.503072 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51774b00-6a8d-4197-8e5e-247af8edbb6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:10.504711 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51774b00-6a8d-4197-8e5e-247af8edbb6d admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:10.516830 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 785/1576] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:10.517879 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51774b00-6a8d-4197-8e5e-247af8edbb6d admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:10.528043 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 792/1577] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:15:10 2022] POST /identity/v3/projects => generated 405 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:10.528720 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1322aa-80b3-48c8-9dfd-9c28b4c50f87 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:10.529740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51774b00-6a8d-4197-8e5e-247af8edbb6d admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:10.548885 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:10.591268 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef9dda5-cc64-4356-aa25-22164e64d49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:10.591884 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef9dda5-cc64-4356-aa25-22164e64d49c admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:10.592170 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef9dda5-cc64-4356-aa25-22164e64d49c admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:10.592452 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef9dda5-cc64-4356-aa25-22164e64d49c admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:10.593667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef9dda5-cc64-4356-aa25-22164e64d49c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:10.595201 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1322aa-80b3-48c8-9dfd-9c28b4c50f87 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:10.595660 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef9dda5-cc64-4356-aa25-22164e64d49c admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:10.600685 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aabfcc5-8091-46ad-a157-05cd21c30f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:10.601346 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aabfcc5-8091-46ad-a157-05cd21c30f3a admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:10.601625 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aabfcc5-8091-46ad-a157-05cd21c30f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:10.601898 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aabfcc5-8091-46ad-a157-05cd21c30f3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:10.602587 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aabfcc5-8091-46ad-a157-05cd21c30f3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:10.604536 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aabfcc5-8091-46ad-a157-05cd21c30f3a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:10.625036 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 786/1578] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:15:10 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:10.626334 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef9dda5-cc64-4356-aa25-22164e64d49c admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:10.634815 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 793/1579] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:15:10 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:10.637157 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfde87-e2ce-495e-8aea-1b0642523977 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:10.637712 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfde87-e2ce-495e-8aea-1b0642523977 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:10.638231 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfde87-e2ce-495e-8aea-1b0642523977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:10.665421 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-49bfde87-e2ce-495e-8aea-1b0642523977 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:10.666645 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-49bfde87-e2ce-495e-8aea-1b0642523977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:10.699219 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef9dda5-cc64-4356-aa25-22164e64d49c admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:10.731692 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 794/1580] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:10.732703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bfde87-e2ce-495e-8aea-1b0642523977 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:10.743747 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bfde87-e2ce-495e-8aea-1b0642523977 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:10.748809 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ace1f-8bc1-4133-8262-077adb31f18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:10.749534 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ace1f-8bc1-4133-8262-077adb31f18e admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:10.749817 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ace1f-8bc1-4133-8262-077adb31f18e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:10.750093 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ace1f-8bc1-4133-8262-077adb31f18e admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:10.750802 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379ace1f-8bc1-4133-8262-077adb31f18e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:10.752858 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379ace1f-8bc1-4133-8262-077adb31f18e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:10.764464 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb36e52-a060-47bf-8d03-897a74f876d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:10.765811 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb36e52-a060-47bf-8d03-897a74f876d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:10.766092 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb36e52-a060-47bf-8d03-897a74f876d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:10.766454 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb36e52-a060-47bf-8d03-897a74f876d8 admin admin] PATH_INFO: `/v3/projects/a01073a6773c4ca0b3208a004cd15c18/users/0fbe204416374bc3b919d14d45d7a653/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:10.784233 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 795/1581] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:15:10 2022] POST /identity/v3/projects => generated 393 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:10.784233 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ace1f-8bc1-4133-8262-077adb31f18e admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:10.784936 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb36e52-a060-47bf-8d03-897a74f876d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a01073a6773c4ca0b3208a004cd15c18, user_id=0fbe204416374bc3b919d14d45d7a653, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:10.787597 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb36e52-a060-47bf-8d03-897a74f876d8 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:10.803189 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 787/1582] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:15:10 2022] PUT /identity/v3/projects/a01073a6773c4ca0b3208a004cd15c18/users/0fbe204416374bc3b919d14d45d7a653/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:10.804165 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb36e52-a060-47bf-8d03-897a74f876d8 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:10.846561 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:10.851757 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:10.862047 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ace1f-8bc1-4133-8262-077adb31f18e admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:10.882326 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb36e52-a060-47bf-8d03-897a74f876d8 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:10.887272 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d4a215-4de0-4f6d-8774-9e112f7a70a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:10.888235 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4a215-4de0-4f6d-8774-9e112f7a70a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:10.888519 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4a215-4de0-4f6d-8774-9e112f7a70a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:10.888799 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4a215-4de0-4f6d-8774-9e112f7a70a3 admin admin] PATH_INFO: `/v3/projects/a21be4dfe67e43d0915559bad38fa60c/users/646d135cd569428c8eda3577df418963/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:10.907382 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d4a215-4de0-4f6d-8774-9e112f7a70a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a21be4dfe67e43d0915559bad38fa60c, user_id=646d135cd569428c8eda3577df418963, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:10.910109 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d4a215-4de0-4f6d-8774-9e112f7a70a3 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:10.925171 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 788/1583] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:15:10 2022] PUT /identity/v3/projects/a21be4dfe67e43d0915559bad38fa60c/users/646d135cd569428c8eda3577df418963/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:10.929945 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d4a215-4de0-4f6d-8774-9e112f7a70a3 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:10.931558 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d2575-cdac-430c-a55e-9139fe8ae0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:10.932693 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d2575-cdac-430c-a55e-9139fe8ae0c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:10.932971 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d2575-cdac-430c-a55e-9139fe8ae0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:10.933248 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d2575-cdac-430c-a55e-9139fe8ae0c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:10.933959 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d2575-cdac-430c-a55e-9139fe8ae0c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:10.935904 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d2575-cdac-430c-a55e-9139fe8ae0c3 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:10.968876 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 796/1584] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:15:10 2022] POST /identity/v3/projects => generated 371 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:10.968876 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d2575-cdac-430c-a55e-9139fe8ae0c3 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:10.992427 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d4a215-4de0-4f6d-8774-9e112f7a70a3 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:11.031622 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d2575-cdac-430c-a55e-9139fe8ae0c3 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:11.036280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5fe098-51f3-42e0-a2a5-7d121554aa2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:11.036937 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5fe098-51f3-42e0-a2a5-7d121554aa2d admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:11.037219 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5fe098-51f3-42e0-a2a5-7d121554aa2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:11.037508 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5fe098-51f3-42e0-a2a5-7d121554aa2d admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:11.038169 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5fe098-51f3-42e0-a2a5-7d121554aa2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:11.040142 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5fe098-51f3-42e0-a2a5-7d121554aa2d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:11.048359 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f57c6-59f4-4ab5-b48d-6668d1b12489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:11.049032 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f57c6-59f4-4ab5-b48d-6668d1b12489 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:11.049312 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f57c6-59f4-4ab5-b48d-6668d1b12489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:11.049588 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f57c6-59f4-4ab5-b48d-6668d1b12489 admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:11.050273 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696f57c6-59f4-4ab5-b48d-6668d1b12489 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:11.051393 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: WARNING py.warnings [None req-696f57c6-59f4-4ab5-b48d-6668d1b12489 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 23:15:11.051393 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: warnings.warn(msg) Jul 13 23:15:11.051393 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: Jul 13 23:15:11.052917 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696f57c6-59f4-4ab5-b48d-6668d1b12489 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:11.060907 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 797/1585] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:15:10 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:11.061816 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5fe098-51f3-42e0-a2a5-7d121554aa2d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:11.071975 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5fe098-51f3-42e0-a2a5-7d121554aa2d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:11.076447 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 789/1586] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:15:10 2022] POST /identity/v3/users => generated 303 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:11.076447 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f57c6-59f4-4ab5-b48d-6668d1b12489 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:11.080232 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec4479-2847-43b2-8d33-8326fdbdea3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:11.080896 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec4479-2847-43b2-8d33-8326fdbdea3e admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:11.081178 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec4479-2847-43b2-8d33-8326fdbdea3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:11.081455 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec4479-2847-43b2-8d33-8326fdbdea3e admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:11.082137 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec4479-2847-43b2-8d33-8326fdbdea3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:11.084133 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec4479-2847-43b2-8d33-8326fdbdea3e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:11.108596 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 798/1587] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:15:11 2022] POST /identity/v3/users => generated 292 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:11.108596 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec4479-2847-43b2-8d33-8326fdbdea3e admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:11.116600 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec4479-2847-43b2-8d33-8326fdbdea3e admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:11.121433 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ebaa4-fde9-4013-baf6-706c85bfb808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:11.122134 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ebaa4-fde9-4013-baf6-706c85bfb808 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:11.122413 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ebaa4-fde9-4013-baf6-706c85bfb808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:11.122736 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ebaa4-fde9-4013-baf6-706c85bfb808 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:11.123450 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ebaa4-fde9-4013-baf6-706c85bfb808 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:11.125455 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ebaa4-fde9-4013-baf6-706c85bfb808 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:11.135367 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f57c6-59f4-4ab5-b48d-6668d1b12489 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:11.140408 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aea620-ff14-4dc9-a58a-cda29565abe6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:11.141087 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aea620-ff14-4dc9-a58a-cda29565abe6 service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:11.141412 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aea620-ff14-4dc9-a58a-cda29565abe6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:11.141858 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aea620-ff14-4dc9-a58a-cda29565abe6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:11.148476 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 799/1588] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:15:11 2022] POST /identity/v3/users => generated 297 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:11.149964 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083eaf7-fedb-49a8-8b11-61e967b914f1 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:11.150260 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083eaf7-fedb-49a8-8b11-61e967b914f1 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:11.150533 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083eaf7-fedb-49a8-8b11-61e967b914f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:11.154235 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aea620-ff14-4dc9-a58a-cda29565abe6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:11.156297 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aea620-ff14-4dc9-a58a-cda29565abe6 service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:11.160478 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:11.167946 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-7083eaf7-fedb-49a8-8b11-61e967b914f1 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:11.168123 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 790/1589] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:15:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:15:11.169168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7083eaf7-fedb-49a8-8b11-61e967b914f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:11.169361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d73ddb0-9a91-4674-9a0d-3772081773c1 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:11.169592 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d73ddb0-9a91-4674-9a0d-3772081773c1 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:11.169879 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d73ddb0-9a91-4674-9a0d-3772081773c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:11.177476 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 800/1590] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:11.178282 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7083eaf7-fedb-49a8-8b11-61e967b914f1 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:11.189803 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-1d73ddb0-9a91-4674-9a0d-3772081773c1 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:11.190352 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7083eaf7-fedb-49a8-8b11-61e967b914f1 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:11.191174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-1d73ddb0-9a91-4674-9a0d-3772081773c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:11.196132 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67893dfc-6722-4345-898a-bc657932e8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:11.196837 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67893dfc-6722-4345-898a-bc657932e8b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:11.197120 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67893dfc-6722-4345-898a-bc657932e8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:11.197401 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67893dfc-6722-4345-898a-bc657932e8b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:11.198087 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67893dfc-6722-4345-898a-bc657932e8b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:11.200215 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67893dfc-6722-4345-898a-bc657932e8b8 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:11.204048 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 791/1591] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:11.205068 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d73ddb0-9a91-4674-9a0d-3772081773c1 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:11.216219 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d73ddb0-9a91-4674-9a0d-3772081773c1 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:11.221825 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b69cf5-afc3-4cb0-8c71-271de3d94aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:11.222489 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b69cf5-afc3-4cb0-8c71-271de3d94aee admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:11.222769 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b69cf5-afc3-4cb0-8c71-271de3d94aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:11.223047 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b69cf5-afc3-4cb0-8c71-271de3d94aee admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:11.223837 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b69cf5-afc3-4cb0-8c71-271de3d94aee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:11.225414 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b69cf5-afc3-4cb0-8c71-271de3d94aee admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:11.226340 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 801/1592] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:15:11 2022] POST /identity/v3/users => generated 286 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:11.227246 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67893dfc-6722-4345-898a-bc657932e8b8 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:11.240156 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67893dfc-6722-4345-898a-bc657932e8b8 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:11.240376 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 792/1593] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:11.240376 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b69cf5-afc3-4cb0-8c71-271de3d94aee admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:11.245429 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87445f18-15ea-4614-bfb4-2c6f9661bc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:11.246238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87445f18-15ea-4614-bfb4-2c6f9661bc3d admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:11.246983 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87445f18-15ea-4614-bfb4-2c6f9661bc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:11.246983 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87445f18-15ea-4614-bfb4-2c6f9661bc3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:11.247838 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87445f18-15ea-4614-bfb4-2c6f9661bc3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:11.250085 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87445f18-15ea-4614-bfb4-2c6f9661bc3d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:11.252352 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b69cf5-afc3-4cb0-8c71-271de3d94aee admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:11.258447 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59837f92-9e77-4947-9dd4-45ed104b8a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:11.259474 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59837f92-9e77-4947-9dd4-45ed104b8a71 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:11.259899 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59837f92-9e77-4947-9dd4-45ed104b8a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:11.260326 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59837f92-9e77-4947-9dd4-45ed104b8a71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:11.261457 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59837f92-9e77-4947-9dd4-45ed104b8a71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:11.262381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 802/1594] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:11.263210 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87445f18-15ea-4614-bfb4-2c6f9661bc3d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:11.263709 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59837f92-9e77-4947-9dd4-45ed104b8a71 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:11.269071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:11.274154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87445f18-15ea-4614-bfb4-2c6f9661bc3d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:11.281088 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 793/1595] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:11.281293 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5b46bb-4ccf-47e0-b779-a16cd50d54cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:11.282158 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b46bb-4ccf-47e0-b779-a16cd50d54cc admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:11.282277 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59837f92-9e77-4947-9dd4-45ed104b8a71 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:11.283097 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b46bb-4ccf-47e0-b779-a16cd50d54cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:11.283427 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b46bb-4ccf-47e0-b779-a16cd50d54cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:11.284225 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5b46bb-4ccf-47e0-b779-a16cd50d54cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:11.284349 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-59837f92-9e77-4947-9dd4-45ed104b8a71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:11.286100 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5b46bb-4ccf-47e0-b779-a16cd50d54cc admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:11.296875 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 803/1596] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:11.297798 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5b46bb-4ccf-47e0-b779-a16cd50d54cc admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:11.298384 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59837f92-9e77-4947-9dd4-45ed104b8a71 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:11.304827 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf77c2-e43e-40a8-9db8-ffea6ab42824 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:11.305675 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf77c2-e43e-40a8-9db8-ffea6ab42824 service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:11.305961 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf77c2-e43e-40a8-9db8-ffea6ab42824 service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:11.306234 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf77c2-e43e-40a8-9db8-ffea6ab42824 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:11.307978 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5b46bb-4ccf-47e0-b779-a16cd50d54cc admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:11.312357 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b4a7d-71f7-4c7e-a84d-14afcc2aab6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:11.312998 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b4a7d-71f7-4c7e-a84d-14afcc2aab6d service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:11.313290 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b4a7d-71f7-4c7e-a84d-14afcc2aab6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:11.313569 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66b4a7d-71f7-4c7e-a84d-14afcc2aab6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:11.317494 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bf77c2-e43e-40a8-9db8-ffea6ab42824 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:11.319053 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bf77c2-e43e-40a8-9db8-ffea6ab42824 service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:11.323246 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b4a7d-71f7-4c7e-a84d-14afcc2aab6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:11.324847 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66b4a7d-71f7-4c7e-a84d-14afcc2aab6d service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:11.331844 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 794/1597] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:15:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:15:11.332583 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf77c2-e43e-40a8-9db8-ffea6ab42824 service neutron] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:11.336359 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 804/1598] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:15:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:15:11.337287 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b4a7d-71f7-4c7e-a84d-14afcc2aab6d service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:11.342915 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf77c2-e43e-40a8-9db8-ffea6ab42824 service neutron] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:11.347909 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66b4a7d-71f7-4c7e-a84d-14afcc2aab6d service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:11.348403 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0e2115-1039-4ead-b814-06e5d5302dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:11.349236 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e2115-1039-4ead-b814-06e5d5302dc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:11.349589 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e2115-1039-4ead-b814-06e5d5302dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:11.349963 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e2115-1039-4ead-b814-06e5d5302dc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:11.350784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0e2115-1039-4ead-b814-06e5d5302dc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:11.353436 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0e2115-1039-4ead-b814-06e5d5302dc7 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:11.353847 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2dbe89-0f9e-4e01-9fee-d5807ab84da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:11.355568 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2dbe89-0f9e-4e01-9fee-d5807ab84da0 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:11.355839 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2dbe89-0f9e-4e01-9fee-d5807ab84da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:11.356186 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2dbe89-0f9e-4e01-9fee-d5807ab84da0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:11.357196 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2dbe89-0f9e-4e01-9fee-d5807ab84da0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:11.359176 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2dbe89-0f9e-4e01-9fee-d5807ab84da0 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:11.365814 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 795/1599] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:11.366647 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0e2115-1039-4ead-b814-06e5d5302dc7 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:11.380960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 805/1600] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:11.382171 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2dbe89-0f9e-4e01-9fee-d5807ab84da0 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:11.398065 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2dbe89-0f9e-4e01-9fee-d5807ab84da0 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:11.399832 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0e2115-1039-4ead-b814-06e5d5302dc7 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:11.404447 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46b51cb-4e5c-4133-98aa-a042968281d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:11.405704 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b51cb-4e5c-4133-98aa-a042968281d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:11.406216 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b51cb-4e5c-4133-98aa-a042968281d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:11.406742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46b51cb-4e5c-4133-98aa-a042968281d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:11.407769 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46b51cb-4e5c-4133-98aa-a042968281d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:11.408050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32ba6f-4ffb-4c87-8671-51dac4ae71db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:11.408724 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32ba6f-4ffb-4c87-8671-51dac4ae71db admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:11.409119 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32ba6f-4ffb-4c87-8671-51dac4ae71db admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:11.409359 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32ba6f-4ffb-4c87-8671-51dac4ae71db admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:11.410100 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc32ba6f-4ffb-4c87-8671-51dac4ae71db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:11.410923 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46b51cb-4e5c-4133-98aa-a042968281d3 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:11.411907 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc32ba6f-4ffb-4c87-8671-51dac4ae71db admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:11.426463 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 796/1601] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:11.426687 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 806/1602] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:11.426793 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32ba6f-4ffb-4c87-8671-51dac4ae71db admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:11.426907 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46b51cb-4e5c-4133-98aa-a042968281d3 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:11.439845 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46b51cb-4e5c-4133-98aa-a042968281d3 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:11.440848 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32ba6f-4ffb-4c87-8671-51dac4ae71db admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:11.445937 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569f4621-e735-486f-949f-45bb3299e28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:11.446693 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f4621-e735-486f-949f-45bb3299e28d admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:11.447105 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f4621-e735-486f-949f-45bb3299e28d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:11.447441 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f4621-e735-486f-949f-45bb3299e28d admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:11.448307 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569f4621-e735-486f-949f-45bb3299e28d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:11.448621 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0573c7-bb6c-4e5e-ab28-f7495844ed86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:11.449526 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0573c7-bb6c-4e5e-ab28-f7495844ed86 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:11.449953 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0573c7-bb6c-4e5e-ab28-f7495844ed86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:11.450237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569f4621-e735-486f-949f-45bb3299e28d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:11.450956 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0573c7-bb6c-4e5e-ab28-f7495844ed86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:11.451927 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0573c7-bb6c-4e5e-ab28-f7495844ed86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:11.454638 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0573c7-bb6c-4e5e-ab28-f7495844ed86 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:11.467077 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 807/1603] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:11.468695 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569f4621-e735-486f-949f-45bb3299e28d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:11.469937 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 797/1604] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:11.471349 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0573c7-bb6c-4e5e-ab28-f7495844ed86 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:11.481287 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569f4621-e735-486f-949f-45bb3299e28d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:11.484963 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0573c7-bb6c-4e5e-ab28-f7495844ed86 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:11.491711 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc36f3d-0f23-4e4b-9e5d-7fcef90a7cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:11.492741 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc132c06-6fab-4261-9fec-355f93cad7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:11.493047 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc36f3d-0f23-4e4b-9e5d-7fcef90a7cb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:11.493559 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc36f3d-0f23-4e4b-9e5d-7fcef90a7cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:11.494048 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc36f3d-0f23-4e4b-9e5d-7fcef90a7cb1 admin admin] PATH_INFO: `/v3/projects/e26fc4a2d5f745ae90c92aeb1cd65f41/users/06685b35e0cd49de98798560fd2bee83/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:11.494356 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc132c06-6fab-4261-9fec-355f93cad7d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:11.494957 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc132c06-6fab-4261-9fec-355f93cad7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:11.495500 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc132c06-6fab-4261-9fec-355f93cad7d4 admin admin] PATH_INFO: `/v3/projects/1ce9fb9212ee408089fd63fb9e9e1de8/users/75652cd3bfa04c7ebae2cbea77cb5fd1/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:11.512016 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc36f3d-0f23-4e4b-9e5d-7fcef90a7cb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e26fc4a2d5f745ae90c92aeb1cd65f41, user_id=06685b35e0cd49de98798560fd2bee83, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:11.515568 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc36f3d-0f23-4e4b-9e5d-7fcef90a7cb1 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:11.526897 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc132c06-6fab-4261-9fec-355f93cad7d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ce9fb9212ee408089fd63fb9e9e1de8, user_id=75652cd3bfa04c7ebae2cbea77cb5fd1, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:11.530252 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: WARNING py.warnings [None req-dc132c06-6fab-4261-9fec-355f93cad7d4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 23:15:11.530252 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: warnings.warn(msg) Jul 13 23:15:11.530252 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: Jul 13 23:15:11.536567 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc132c06-6fab-4261-9fec-355f93cad7d4 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:11.547585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 798/1605] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:15:11 2022] PUT /identity/v3/projects/e26fc4a2d5f745ae90c92aeb1cd65f41/users/06685b35e0cd49de98798560fd2bee83/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:11.547585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc36f3d-0f23-4e4b-9e5d-7fcef90a7cb1 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:11.569262 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 808/1606] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:15:11 2022] PUT /identity/v3/projects/1ce9fb9212ee408089fd63fb9e9e1de8/users/75652cd3bfa04c7ebae2cbea77cb5fd1/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:11.569995 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc132c06-6fab-4261-9fec-355f93cad7d4 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:11.645907 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc36f3d-0f23-4e4b-9e5d-7fcef90a7cb1 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:11.704603 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc132c06-6fab-4261-9fec-355f93cad7d4 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:11.714602 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd5b52-a80d-4cef-894c-3c8bfb88cbc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:11.715885 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd5b52-a80d-4cef-894c-3c8bfb88cbc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:11.716274 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd5b52-a80d-4cef-894c-3c8bfb88cbc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:11.717100 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd5b52-a80d-4cef-894c-3c8bfb88cbc8 admin admin] PATH_INFO: `/v3/projects/91d9bdd19b054dc68ac2fd70021390cf/users/f624cdb8f49f4eaaaea665afe0a72602/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:11.728464 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd461f6e-99a6-4b65-ac5a-e42a445a8996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:11.730054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd461f6e-99a6-4b65-ac5a-e42a445a8996 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:11.730054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd461f6e-99a6-4b65-ac5a-e42a445a8996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:11.730292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd461f6e-99a6-4b65-ac5a-e42a445a8996 admin admin] PATH_INFO: `/v3/projects/29d1268531c344eaaf28dc6c1498a467/users/5e052654361f4580ab36a347610f63c4/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:11.746462 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:11.754296 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd461f6e-99a6-4b65-ac5a-e42a445a8996 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29d1268531c344eaaf28dc6c1498a467, user_id=5e052654361f4580ab36a347610f63c4, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:11.756481 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfd5b52-a80d-4cef-894c-3c8bfb88cbc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91d9bdd19b054dc68ac2fd70021390cf, user_id=f624cdb8f49f4eaaaea665afe0a72602, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:11.758445 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd461f6e-99a6-4b65-ac5a-e42a445a8996 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:11.763394 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfd5b52-a80d-4cef-894c-3c8bfb88cbc8 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:11.789796 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 799/1607] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:15:11 2022] PUT /identity/v3/projects/29d1268531c344eaaf28dc6c1498a467/users/5e052654361f4580ab36a347610f63c4/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:11.789796 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd461f6e-99a6-4b65-ac5a-e42a445a8996 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:11.806542 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 809/1608] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:15:11 2022] PUT /identity/v3/projects/91d9bdd19b054dc68ac2fd70021390cf/users/f624cdb8f49f4eaaaea665afe0a72602/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:11.806542 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84006916-8a42-4d8c-befa-4cb5e826f1a8 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:11.806542 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84006916-8a42-4d8c-befa-4cb5e826f1a8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:11.806542 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84006916-8a42-4d8c-befa-4cb5e826f1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:11.841653 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-84006916-8a42-4d8c-befa-4cb5e826f1a8 None None] MFA Rules not processed for user `06685b35e0cd49de98798560fd2bee83`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:11.844032 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-84006916-8a42-4d8c-befa-4cb5e826f1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:11.896956 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd461f6e-99a6-4b65-ac5a-e42a445a8996 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:11.920161 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:11.940191 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 810/1609] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:11 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:11.942384 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8257d-ebcd-4a57-a3e7-f70388614aa7 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:11.942855 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8257d-ebcd-4a57-a3e7-f70388614aa7 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:11.943045 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8257d-ebcd-4a57-a3e7-f70388614aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:11.972049 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-d5b8257d-ebcd-4a57-a3e7-f70388614aa7 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:11.973947 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d5b8257d-ebcd-4a57-a3e7-f70388614aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:11.994655 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca383bb-908c-43da-ad4d-aa86f77a12f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:11.995826 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca383bb-908c-43da-ad4d-aa86f77a12f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:11.997264 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca383bb-908c-43da-ad4d-aa86f77a12f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:11.997643 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca383bb-908c-43da-ad4d-aa86f77a12f8 admin admin] PATH_INFO: `/v3/projects/dc01ed4482c04899b1509a655c9c7e3a/users/e2e2bd6cf0a44d529ef9b2678c467aba/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:12.019890 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca383bb-908c-43da-ad4d-aa86f77a12f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc01ed4482c04899b1509a655c9c7e3a, user_id=e2e2bd6cf0a44d529ef9b2678c467aba, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:12.023013 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca383bb-908c-43da-ad4d-aa86f77a12f8 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:12.047926 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 800/1610] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:15:11 2022] PUT /identity/v3/projects/dc01ed4482c04899b1509a655c9c7e3a/users/e2e2bd6cf0a44d529ef9b2678c467aba/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:12.047926 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc8bd6-8f09-43d2-b4a6-206a5e3ed826 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:12.047926 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc8bd6-8f09-43d2-b4a6-206a5e3ed826 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:12.047926 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc8bd6-8f09-43d2-b4a6-206a5e3ed826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:12.059370 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 811/1611] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:12.061363 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22baefe-1a14-4f42-90a7-9d07eeab8b37 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:12.061683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22baefe-1a14-4f42-90a7-9d07eeab8b37 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:12.062184 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22baefe-1a14-4f42-90a7-9d07eeab8b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:12.074661 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-73fc8bd6-8f09-43d2-b4a6-206a5e3ed826 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:12.076308 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-73fc8bd6-8f09-43d2-b4a6-206a5e3ed826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:12.101111 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-d22baefe-1a14-4f42-90a7-9d07eeab8b37 None None] MFA Rules not processed for user `f624cdb8f49f4eaaaea665afe0a72602`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:12.102550 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d22baefe-1a14-4f42-90a7-9d07eeab8b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:12.164189 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 801/1612] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:12.165384 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc8bd6-8f09-43d2-b4a6-206a5e3ed826 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:12.176808 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc8bd6-8f09-43d2-b4a6-206a5e3ed826 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:12.183906 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804b697-ddc9-4c06-aee6-24d161fa2574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:12.184668 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804b697-ddc9-4c06-aee6-24d161fa2574 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:12.185308 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804b697-ddc9-4c06-aee6-24d161fa2574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:12.185648 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804b697-ddc9-4c06-aee6-24d161fa2574 admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:12.186436 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7804b697-ddc9-4c06-aee6-24d161fa2574 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:12.189016 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7804b697-ddc9-4c06-aee6-24d161fa2574 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:12.221012 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:12.234595 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 802/1613] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:15:12 2022] POST /identity/v3/projects => generated 405 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:12.234595 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c739d-f4cc-407e-a9a2-8f072ff0e39c None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:12.234595 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c739d-f4cc-407e-a9a2-8f072ff0e39c None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:12.234595 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c739d-f4cc-407e-a9a2-8f072ff0e39c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:12.261211 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 812/1614] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:12.268150 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22baefe-1a14-4f42-90a7-9d07eeab8b37 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:12.291142 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-df4c739d-f4cc-407e-a9a2-8f072ff0e39c None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:12.295825 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-df4c739d-f4cc-407e-a9a2-8f072ff0e39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:12.391580 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22baefe-1a14-4f42-90a7-9d07eeab8b37 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:12.451256 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 803/1615] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:12.459148 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c739d-f4cc-407e-a9a2-8f072ff0e39c None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:12.468504 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:12.488225 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0603efc-3fff-4907-9c4f-02c48a7b8e70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:12.489080 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0603efc-3fff-4907-9c4f-02c48a7b8e70 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:12.489464 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0603efc-3fff-4907-9c4f-02c48a7b8e70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:12.489858 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0603efc-3fff-4907-9c4f-02c48a7b8e70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:12.511016 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0603efc-3fff-4907-9c4f-02c48a7b8e70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:12.513647 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0603efc-3fff-4907-9c4f-02c48a7b8e70 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:12.541060 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 813/1616] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:15:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:15:12.542192 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0603efc-3fff-4907-9c4f-02c48a7b8e70 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:12.556384 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0603efc-3fff-4907-9c4f-02c48a7b8e70 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:12.563214 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f73fb9-dae1-4b4e-97af-28d61dd89627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:12.567937 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f73fb9-dae1-4b4e-97af-28d61dd89627 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:12.568242 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f73fb9-dae1-4b4e-97af-28d61dd89627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:12.568920 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f73fb9-dae1-4b4e-97af-28d61dd89627 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:12.569375 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f73fb9-dae1-4b4e-97af-28d61dd89627 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:12.572901 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f73fb9-dae1-4b4e-97af-28d61dd89627 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:12.601596 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c739d-f4cc-407e-a9a2-8f072ff0e39c None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:12.611651 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b1504-296a-4c66-8a44-48303f37b6e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:12.615904 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b1504-296a-4c66-8a44-48303f37b6e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:12.616269 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b1504-296a-4c66-8a44-48303f37b6e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:12.617717 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b1504-296a-4c66-8a44-48303f37b6e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:12.624331 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 814/1617] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:15:12 2022] POST /identity/v3/users => generated 309 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:12.625380 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f73fb9-dae1-4b4e-97af-28d61dd89627 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:12.646300 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9b1504-296a-4c66-8a44-48303f37b6e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:12.652204 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9b1504-296a-4c66-8a44-48303f37b6e6 service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:12.668022 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f73fb9-dae1-4b4e-97af-28d61dd89627 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:12.688294 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edcf55c-d6b0-4c7c-92d8-5f0da377895d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:12.689997 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edcf55c-d6b0-4c7c-92d8-5f0da377895d admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:12.691388 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edcf55c-d6b0-4c7c-92d8-5f0da377895d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:12.691767 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edcf55c-d6b0-4c7c-92d8-5f0da377895d admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:12.697488 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edcf55c-d6b0-4c7c-92d8-5f0da377895d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:12.703969 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edcf55c-d6b0-4c7c-92d8-5f0da377895d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:12.718490 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 804/1618] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:15:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:15:12.719905 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b1504-296a-4c66-8a44-48303f37b6e6 service neutron] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:12.731820 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b1504-296a-4c66-8a44-48303f37b6e6 service neutron] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:12.739404 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c9bc4b-2382-4e59-82dd-a9a7e30b07bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:12.740654 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9bc4b-2382-4e59-82dd-a9a7e30b07bc service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:12.741092 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9bc4b-2382-4e59-82dd-a9a7e30b07bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:12.743850 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9bc4b-2382-4e59-82dd-a9a7e30b07bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:12.751821 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 815/1619] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:15:12 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:12.754104 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edcf55c-d6b0-4c7c-92d8-5f0da377895d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:12.761307 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c9bc4b-2382-4e59-82dd-a9a7e30b07bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:12.763802 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c9bc4b-2382-4e59-82dd-a9a7e30b07bc service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:12.785210 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 805/1620] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:15:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:15:12.787484 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c9bc4b-2382-4e59-82dd-a9a7e30b07bc service neutron] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:12.805229 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c9bc4b-2382-4e59-82dd-a9a7e30b07bc service neutron] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:12.846446 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edcf55c-d6b0-4c7c-92d8-5f0da377895d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:12.854534 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad0f608-76e6-4cc9-8aa0-973852dc9ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:12.855642 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0f608-76e6-4cc9-8aa0-973852dc9ee5 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:12.856390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0f608-76e6-4cc9-8aa0-973852dc9ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:12.856980 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0f608-76e6-4cc9-8aa0-973852dc9ee5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:12.859431 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad0f608-76e6-4cc9-8aa0-973852dc9ee5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:12.862717 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad0f608-76e6-4cc9-8aa0-973852dc9ee5 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:12.877916 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 816/1621] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:12.882183 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad0f608-76e6-4cc9-8aa0-973852dc9ee5 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:12.895521 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad0f608-76e6-4cc9-8aa0-973852dc9ee5 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:12.901456 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e629ec3d-7226-46fb-b4cc-485386b756ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:12.903037 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629ec3d-7226-46fb-b4cc-485386b756ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:12.903472 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629ec3d-7226-46fb-b4cc-485386b756ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:12.904090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629ec3d-7226-46fb-b4cc-485386b756ce admin admin] PATH_INFO: `/v3/projects/5a9434e6932c446b8f3ca24db5526358/users/53007506583449e8a2ad2f132549f8d7/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:12.925519 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e629ec3d-7226-46fb-b4cc-485386b756ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a9434e6932c446b8f3ca24db5526358, user_id=53007506583449e8a2ad2f132549f8d7, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:12.928794 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e629ec3d-7226-46fb-b4cc-485386b756ce admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:12.930003 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772df84b-f160-4b39-9c0b-47aa90b41e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:12.931622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772df84b-f160-4b39-9c0b-47aa90b41e9a admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:12.932626 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772df84b-f160-4b39-9c0b-47aa90b41e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:12.933343 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772df84b-f160-4b39-9c0b-47aa90b41e9a admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:12.934799 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772df84b-f160-4b39-9c0b-47aa90b41e9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:12.938678 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772df84b-f160-4b39-9c0b-47aa90b41e9a admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:12.948155 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 817/1622] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:15:12 2022] PUT /identity/v3/projects/5a9434e6932c446b8f3ca24db5526358/users/53007506583449e8a2ad2f132549f8d7/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:12.953822 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e629ec3d-7226-46fb-b4cc-485386b756ce admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:12.979956 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 806/1623] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:15:12 2022] POST /identity/v3/users => generated 293 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:12.981627 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772df84b-f160-4b39-9c0b-47aa90b41e9a admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:12.997261 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772df84b-f160-4b39-9c0b-47aa90b41e9a admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:13.016084 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:13.079192 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e629ec3d-7226-46fb-b4cc-485386b756ce admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:13.086656 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f652c9-ae50-4f0a-a573-29f01c2d55b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:13.087563 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f652c9-ae50-4f0a-a573-29f01c2d55b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:13.088162 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f652c9-ae50-4f0a-a573-29f01c2d55b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:13.088552 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f652c9-ae50-4f0a-a573-29f01c2d55b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:13.090208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f652c9-ae50-4f0a-a573-29f01c2d55b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:13.092494 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f652c9-ae50-4f0a-a573-29f01c2d55b9 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:13.102242 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69fcde3-22d4-4652-8386-98e3613cc1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:13.102985 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69fcde3-22d4-4652-8386-98e3613cc1ca admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:13.103274 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69fcde3-22d4-4652-8386-98e3613cc1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:13.103560 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69fcde3-22d4-4652-8386-98e3613cc1ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:13.106526 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69fcde3-22d4-4652-8386-98e3613cc1ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:13.107214 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 818/1624] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:15:13.111659 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f652c9-ae50-4f0a-a573-29f01c2d55b9 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:13.112364 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69fcde3-22d4-4652-8386-98e3613cc1ca admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:13.130394 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 807/1625] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:13.131667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69fcde3-22d4-4652-8386-98e3613cc1ca admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:13.136495 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f652c9-ae50-4f0a-a573-29f01c2d55b9 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:13.145211 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69fcde3-22d4-4652-8386-98e3613cc1ca admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:13.151232 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f065d074-0792-4a5c-bb44-f9cd28451e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:13.152272 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b5af6-95da-443d-82cf-2318461ec241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:13.152513 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f065d074-0792-4a5c-bb44-f9cd28451e6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:13.152832 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f065d074-0792-4a5c-bb44-f9cd28451e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:13.153163 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f065d074-0792-4a5c-bb44-f9cd28451e6c admin admin] PATH_INFO: `/v3/projects/7ed9eaf07b7149ed96f7e8df74034602/users/92ca758add884846a1513b353a9f6572/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:13.153276 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b5af6-95da-443d-82cf-2318461ec241 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:13.153463 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b5af6-95da-443d-82cf-2318461ec241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:13.153709 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b5af6-95da-443d-82cf-2318461ec241 admin admin] PATH_INFO: `/v3/projects/5a9434e6932c446b8f3ca24db5526358/users/53007506583449e8a2ad2f132549f8d7/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:13.167113 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556b5af6-95da-443d-82cf-2318461ec241 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a9434e6932c446b8f3ca24db5526358, user_id=53007506583449e8a2ad2f132549f8d7, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:13.167113 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556b5af6-95da-443d-82cf-2318461ec241 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:13.175841 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f065d074-0792-4a5c-bb44-f9cd28451e6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ed9eaf07b7149ed96f7e8df74034602, user_id=92ca758add884846a1513b353a9f6572, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:13.179347 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f065d074-0792-4a5c-bb44-f9cd28451e6c admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:13.180248 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:13.195918 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 819/1626] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:15:13 2022] PUT /identity/v3/projects/5a9434e6932c446b8f3ca24db5526358/users/53007506583449e8a2ad2f132549f8d7/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:13.197927 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c1417-c5a3-4cdb-82d2-bc24c9f11d4b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:13.198060 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c1417-c5a3-4cdb-82d2-bc24c9f11d4b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:13.198275 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c1417-c5a3-4cdb-82d2-bc24c9f11d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:13.229415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 808/1627] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:15:13 2022] PUT /identity/v3/projects/7ed9eaf07b7149ed96f7e8df74034602/users/92ca758add884846a1513b353a9f6572/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:13.230772 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82471963-d446-4b70-94a1-252d69f4e61e None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:13.231177 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82471963-d446-4b70-94a1-252d69f4e61e None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:13.231466 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82471963-d446-4b70-94a1-252d69f4e61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:13.266381 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-82471963-d446-4b70-94a1-252d69f4e61e None None] MFA Rules not processed for user `53007506583449e8a2ad2f132549f8d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:13.267698 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-82471963-d446-4b70-94a1-252d69f4e61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:13.273108 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-cf8c1417-c5a3-4cdb-82d2-bc24c9f11d4b None None] MFA Rules not processed for user `0fbe204416374bc3b919d14d45d7a653`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:13.276266 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-cf8c1417-c5a3-4cdb-82d2-bc24c9f11d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:13.354440 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 809/1628] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:13 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:13.366642 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82471963-d446-4b70-94a1-252d69f4e61e None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:13.455081 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82471963-d446-4b70-94a1-252d69f4e61e None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:13.455744 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 820/1629] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:13 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:13.455744 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c1417-c5a3-4cdb-82d2-bc24c9f11d4b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:13.457756 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a5c57-6a40-4ecb-95da-ffa7994d5def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:13.458559 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a5c57-6a40-4ecb-95da-ffa7994d5def admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:13.458845 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a5c57-6a40-4ecb-95da-ffa7994d5def admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:13.459121 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a5c57-6a40-4ecb-95da-ffa7994d5def admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:13.460059 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703a5c57-6a40-4ecb-95da-ffa7994d5def admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:13.461962 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703a5c57-6a40-4ecb-95da-ffa7994d5def admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:13.480306 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 810/1630] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:13.481281 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a5c57-6a40-4ecb-95da-ffa7994d5def admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:13.497339 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a5c57-6a40-4ecb-95da-ffa7994d5def admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:13.505201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30260996-54f2-4813-9d0b-187982a28bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:13.506701 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30260996-54f2-4813-9d0b-187982a28bee admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:13.507521 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30260996-54f2-4813-9d0b-187982a28bee admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:13.507894 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30260996-54f2-4813-9d0b-187982a28bee admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:13.509681 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30260996-54f2-4813-9d0b-187982a28bee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:13.511852 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30260996-54f2-4813-9d0b-187982a28bee admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:13.526231 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:13.555846 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 811/1631] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:15:13 2022] POST /identity/v3/projects => generated 405 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:13.555846 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30260996-54f2-4813-9d0b-187982a28bee admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:13.556798 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c1417-c5a3-4cdb-82d2-bc24c9f11d4b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:13.660086 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30260996-54f2-4813-9d0b-187982a28bee admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:13.666687 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df7660-4ce1-4fd6-8d8e-e63170e9e8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:13.668114 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df7660-4ce1-4fd6-8d8e-e63170e9e8fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:13.670113 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df7660-4ce1-4fd6-8d8e-e63170e9e8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:13.670483 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df7660-4ce1-4fd6-8d8e-e63170e9e8fc admin admin] PATH_INFO: `/v3/projects/7ed9eaf07b7149ed96f7e8df74034602/users/92ca758add884846a1513b353a9f6572/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:13.676181 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1df7660-4ce1-4fd6-8d8e-e63170e9e8fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ed9eaf07b7149ed96f7e8df74034602, user_id=92ca758add884846a1513b353a9f6572, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:13.677139 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c8260-853a-493f-b62d-73003f19ad49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:13.677793 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c8260-853a-493f-b62d-73003f19ad49 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:13.678071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c8260-853a-493f-b62d-73003f19ad49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:13.678342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c8260-853a-493f-b62d-73003f19ad49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:13.680186 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1df7660-4ce1-4fd6-8d8e-e63170e9e8fc admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:13.703878 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 812/1632] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:15:13 2022] PUT /identity/v3/projects/7ed9eaf07b7149ed96f7e8df74034602/users/92ca758add884846a1513b353a9f6572/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:13.705527 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df7660-4ce1-4fd6-8d8e-e63170e9e8fc admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:13.805528 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:13.812798 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c8260-853a-493f-b62d-73003f19ad49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:13.814368 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c8260-853a-493f-b62d-73003f19ad49 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:13.840401 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 821/1633] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:15:13 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:15:13.841756 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ddc2e-691e-41b9-8e80-b904b0a768fe None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:13.842036 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ddc2e-691e-41b9-8e80-b904b0a768fe None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:13.842306 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ddc2e-691e-41b9-8e80-b904b0a768fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:13.858429 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df7660-4ce1-4fd6-8d8e-e63170e9e8fc admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:13.869425 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8e72ff-49df-436e-b212-cd0b83594fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:13.871136 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e72ff-49df-436e-b212-cd0b83594fe7 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:13.871136 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e72ff-49df-436e-b212-cd0b83594fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:13.871369 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e72ff-49df-436e-b212-cd0b83594fe7 admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:13.875472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8e72ff-49df-436e-b212-cd0b83594fe7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:13.877732 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8e72ff-49df-436e-b212-cd0b83594fe7 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:13.897816 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-984ddc2e-691e-41b9-8e80-b904b0a768fe None None] MFA Rules not processed for user `92ca758add884846a1513b353a9f6572`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:13.898843 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-984ddc2e-691e-41b9-8e80-b904b0a768fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:13.915873 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 813/1634] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:15:13 2022] POST /identity/v3/users => generated 309 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:13.919861 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8e72ff-49df-436e-b212-cd0b83594fe7 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:13.947619 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8e72ff-49df-436e-b212-cd0b83594fe7 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:13.960165 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7178998d-ec39-40c2-9bdc-d6063ac0ddcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:13.961164 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178998d-ec39-40c2-9bdc-d6063ac0ddcb admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:13.961640 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178998d-ec39-40c2-9bdc-d6063ac0ddcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:13.962103 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178998d-ec39-40c2-9bdc-d6063ac0ddcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:13.963044 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7178998d-ec39-40c2-9bdc-d6063ac0ddcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:13.964956 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7178998d-ec39-40c2-9bdc-d6063ac0ddcb admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:13.982031 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 822/1635] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:13 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:13.994491 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ddc2e-691e-41b9-8e80-b904b0a768fe None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:14.010396 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 814/1636] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:14.011479 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7178998d-ec39-40c2-9bdc-d6063ac0ddcb admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:14.030957 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ddc2e-691e-41b9-8e80-b904b0a768fe None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:14.042693 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f87a757-bcee-45d2-a1ce-295f6b11a2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:14.050071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f87a757-bcee-45d2-a1ce-295f6b11a2f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:14.051946 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f87a757-bcee-45d2-a1ce-295f6b11a2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:14.052061 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f87a757-bcee-45d2-a1ce-295f6b11a2f2 admin admin] PATH_INFO: `/v3/projects/f7678a8dfe21460184a8ab7206e3244d/users/49e2ce186d9b421fbccdcaa67c030c9a/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:14.128414 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f87a757-bcee-45d2-a1ce-295f6b11a2f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7678a8dfe21460184a8ab7206e3244d, user_id=49e2ce186d9b421fbccdcaa67c030c9a, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:14.151782 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f87a757-bcee-45d2-a1ce-295f6b11a2f2 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:14.199749 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 823/1637] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:15:13 2022] PUT /identity/v3/projects/f7678a8dfe21460184a8ab7206e3244d/users/49e2ce186d9b421fbccdcaa67c030c9a/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:15:14.204727 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f87a757-bcee-45d2-a1ce-295f6b11a2f2 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:14.258034 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7178998d-ec39-40c2-9bdc-d6063ac0ddcb admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:14.309886 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:14.401883 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f87a757-bcee-45d2-a1ce-295f6b11a2f2 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:14.403967 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f2571-048c-4471-bdae-1d764e168d3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:14.407313 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f2571-048c-4471-bdae-1d764e168d3a service cinder] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:14.407624 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f2571-048c-4471-bdae-1d764e168d3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:14.407902 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f2571-048c-4471-bdae-1d764e168d3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:14.417782 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9145e64d-01d2-4d0a-aec7-df77863dd33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:14.418475 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9145e64d-01d2-4d0a-aec7-df77863dd33a admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:14.418762 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9145e64d-01d2-4d0a-aec7-df77863dd33a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:14.419042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9145e64d-01d2-4d0a-aec7-df77863dd33a admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:14.419812 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9145e64d-01d2-4d0a-aec7-df77863dd33a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:14.422006 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9145e64d-01d2-4d0a-aec7-df77863dd33a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:14.449948 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 824/1638] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:14.449948 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9145e64d-01d2-4d0a-aec7-df77863dd33a admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:14.465382 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9145e64d-01d2-4d0a-aec7-df77863dd33a admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:14.472826 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf02f787-22e3-4817-bb55-ab27040c47a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:14.473646 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02f787-22e3-4817-bb55-ab27040c47a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:14.473930 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02f787-22e3-4817-bb55-ab27040c47a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:14.474539 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02f787-22e3-4817-bb55-ab27040c47a4 admin admin] PATH_INFO: `/v3/users/06685b35e0cd49de98798560fd2bee83` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:14.476798 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf02f787-22e3-4817-bb55-ab27040c47a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06685b35e0cd49de98798560fd2bee83)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:14.478128 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: WARNING py.warnings [None req-bf02f787-22e3-4817-bb55-ab27040c47a4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 23:15:14.478128 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: warnings.warn(msg) Jul 13 23:15:14.478128 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: Jul 13 23:15:14.480860 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf02f787-22e3-4817-bb55-ab27040c47a4 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:14.562421 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02f2571-048c-4471-bdae-1d764e168d3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:14.564537 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02f2571-048c-4471-bdae-1d764e168d3a service cinder] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:14.576375 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 815/1639] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:15:14 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:15:14.577473 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f2571-048c-4471-bdae-1d764e168d3a service cinder] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:14.589564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f2571-048c-4471-bdae-1d764e168d3a service cinder] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:14.595417 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:14.596533 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4516af10-bcec-4146-8877-54c65f4cd8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:14.597379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516af10-bcec-4146-8877-54c65f4cd8be admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:14.597633 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516af10-bcec-4146-8877-54c65f4cd8be admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:14.597951 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516af10-bcec-4146-8877-54c65f4cd8be admin admin] PATH_INFO: `/v3/projects/f7678a8dfe21460184a8ab7206e3244d/users/49e2ce186d9b421fbccdcaa67c030c9a/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:14.599965 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-bf02f787-22e3-4817-bb55-ab27040c47a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06685b35e0cd49de98798560fd2bee83'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:14.600946 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4516af10-bcec-4146-8877-54c65f4cd8be admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7678a8dfe21460184a8ab7206e3244d, user_id=49e2ce186d9b421fbccdcaa67c030c9a, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:14.602143 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-bf02f787-22e3-4817-bb55-ab27040c47a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06685b35e0cd49de98798560fd2bee83'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:14.604983 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4516af10-bcec-4146-8877-54c65f4cd8be admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:14.637695 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 816/1640] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:15:14 2022] PUT /identity/v3/projects/f7678a8dfe21460184a8ab7206e3244d/users/49e2ce186d9b421fbccdcaa67c030c9a/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:14.645132 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-bf02f787-22e3-4817-bb55-ab27040c47a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06685b35e0cd49de98798560fd2bee83'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:14.659351 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-bf02f787-22e3-4817-bb55-ab27040c47a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06685b35e0cd49de98798560fd2bee83'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:14.677383 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-bf02f787-22e3-4817-bb55-ab27040c47a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06685b35e0cd49de98798560fd2bee83'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:14.694343 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 825/1641] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:15:14 2022] DELETE /identity/v3/users/06685b35e0cd49de98798560fd2bee83 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:14.704912 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8740d40-093b-447f-ac7e-1acaa6e0581a None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:14.706702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8740d40-093b-447f-ac7e-1acaa6e0581a None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:14.707837 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8740d40-093b-447f-ac7e-1acaa6e0581a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:14.709617 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf02f787-22e3-4817-bb55-ab27040c47a4 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:14.711369 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-bf02f787-22e3-4817-bb55-ab27040c47a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:14.745007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-c8740d40-093b-447f-ac7e-1acaa6e0581a None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:14.747053 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-c8740d40-093b-447f-ac7e-1acaa6e0581a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:14.829131 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf02f787-22e3-4817-bb55-ab27040c47a4 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:14.833566 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358c8a2-54ea-4648-840e-a62236e7f0e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:14.834368 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358c8a2-54ea-4648-840e-a62236e7f0e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:14.835085 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358c8a2-54ea-4648-840e-a62236e7f0e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:14.835225 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358c8a2-54ea-4648-840e-a62236e7f0e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:14.838650 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4358c8a2-54ea-4648-840e-a62236e7f0e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:14.871746 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 817/1642] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:14.904362 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8740d40-093b-447f-ac7e-1acaa6e0581a None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:14.907304 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-c8740d40-093b-447f-ac7e-1acaa6e0581a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:15.005325 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4358c8a2-54ea-4648-840e-a62236e7f0e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:15.008242 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4358c8a2-54ea-4648-840e-a62236e7f0e3 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:15.043926 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 826/1643] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:15:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:15:15.059746 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8740d40-093b-447f-ac7e-1acaa6e0581a None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:15.071232 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac2431-7e71-4b56-ab4c-a2b006f810e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:15.072023 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac2431-7e71-4b56-ab4c-a2b006f810e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:15.072307 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac2431-7e71-4b56-ab4c-a2b006f810e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:15.072635 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac2431-7e71-4b56-ab4c-a2b006f810e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:15.095235 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cac2431-7e71-4b56-ab4c-a2b006f810e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:15.097191 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cac2431-7e71-4b56-ab4c-a2b006f810e3 service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:15.109883 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:15.129855 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 818/1644] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:15:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:15:16.484243 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358c8a2-54ea-4648-840e-a62236e7f0e3 service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:16.486275 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4358c8a2-54ea-4648-840e-a62236e7f0e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:16.536571 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358c8a2-54ea-4648-840e-a62236e7f0e3 service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:16.538093 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0497b889-d7e6-4fda-a49e-8517b4160bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:16.539569 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0497b889-d7e6-4fda-a49e-8517b4160bfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:16.539569 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0497b889-d7e6-4fda-a49e-8517b4160bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:16.539569 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0497b889-d7e6-4fda-a49e-8517b4160bfd admin admin] PATH_INFO: `/v3/projects/e26fc4a2d5f745ae90c92aeb1cd65f41` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:16.545681 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0497b889-d7e6-4fda-a49e-8517b4160bfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e26fc4a2d5f745ae90c92aeb1cd65f41)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:16.555400 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0497b889-d7e6-4fda-a49e-8517b4160bfd admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:16.665887 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0497b889-d7e6-4fda-a49e-8517b4160bfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e26fc4a2d5f745ae90c92aeb1cd65f41', 'request_id': 'req-0497b889-d7e6-4fda-a49e-8517b4160bfd'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:16.678030 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0497b889-d7e6-4fda-a49e-8517b4160bfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e26fc4a2d5f745ae90c92aeb1cd65f41', 'request_id': 'req-0497b889-d7e6-4fda-a49e-8517b4160bfd'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:16.691552 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0497b889-d7e6-4fda-a49e-8517b4160bfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e26fc4a2d5f745ae90c92aeb1cd65f41', 'request_id': 'req-0497b889-d7e6-4fda-a49e-8517b4160bfd'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:16.704247 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:16.714282 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0497b889-d7e6-4fda-a49e-8517b4160bfd admin admin] The token cache is being invalidate because project e26fc4a2d5f745ae90c92aeb1cd65f41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:15:16.714595 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0497b889-d7e6-4fda-a49e-8517b4160bfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:16.718957 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0497b889-d7e6-4fda-a49e-8517b4160bfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:16.721487 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 827/1645] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:15:16 2022] DELETE /identity/v3/projects/e26fc4a2d5f745ae90c92aeb1cd65f41 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:16.725566 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac2431-7e71-4b56-ab4c-a2b006f810e3 service neutron] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:16.740307 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac2431-7e71-4b56-ab4c-a2b006f810e3 service neutron] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:16.887155 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da42cc3-6fef-42cd-97ae-7a03cc704fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:16.888030 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da42cc3-6fef-42cd-97ae-7a03cc704fa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:16.888580 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da42cc3-6fef-42cd-97ae-7a03cc704fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:16.888710 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da42cc3-6fef-42cd-97ae-7a03cc704fa4 admin admin] PATH_INFO: `/v3/users/53007506583449e8a2ad2f132549f8d7` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:16.892556 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da42cc3-6fef-42cd-97ae-7a03cc704fa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53007506583449e8a2ad2f132549f8d7)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:16.894395 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da42cc3-6fef-42cd-97ae-7a03cc704fa4 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:17.025744 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-6da42cc3-6fef-42cd-97ae-7a03cc704fa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53007506583449e8a2ad2f132549f8d7'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:17.026444 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-6da42cc3-6fef-42cd-97ae-7a03cc704fa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53007506583449e8a2ad2f132549f8d7'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:17.068508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-6da42cc3-6fef-42cd-97ae-7a03cc704fa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53007506583449e8a2ad2f132549f8d7'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:17.091015 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-6da42cc3-6fef-42cd-97ae-7a03cc704fa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53007506583449e8a2ad2f132549f8d7'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:17.092922 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-6da42cc3-6fef-42cd-97ae-7a03cc704fa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53007506583449e8a2ad2f132549f8d7'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:17.138515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 819/1646] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:15:16 2022] DELETE /identity/v3/users/53007506583449e8a2ad2f132549f8d7 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:18.633857 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61aa4a5-da2a-431d-8845-52150f36d270 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:18.634689 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61aa4a5-da2a-431d-8845-52150f36d270 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:18.635135 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61aa4a5-da2a-431d-8845-52150f36d270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:18.677441 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-a61aa4a5-da2a-431d-8845-52150f36d270 None None] MFA Rules not processed for user `49e2ce186d9b421fbccdcaa67c030c9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:18.679182 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a61aa4a5-da2a-431d-8845-52150f36d270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:18.760510 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 828/1647] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:18 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:19.297414 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da42cc3-6fef-42cd-97ae-7a03cc704fa4 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:19.299849 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-6da42cc3-6fef-42cd-97ae-7a03cc704fa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:19.458422 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da42cc3-6fef-42cd-97ae-7a03cc704fa4 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:19.467380 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e5b2e7-400c-45da-9dd5-c6fe5cb60f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:19.469416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5b2e7-400c-45da-9dd5-c6fe5cb60f78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:19.469956 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5b2e7-400c-45da-9dd5-c6fe5cb60f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:19.470543 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5b2e7-400c-45da-9dd5-c6fe5cb60f78 admin admin] PATH_INFO: `/v3/projects/5a9434e6932c446b8f3ca24db5526358` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:19.472401 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e5b2e7-400c-45da-9dd5-c6fe5cb60f78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a9434e6932c446b8f3ca24db5526358)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:19.474152 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: WARNING py.warnings [None req-62e5b2e7-400c-45da-9dd5-c6fe5cb60f78 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 23:15:19.474152 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: warnings.warn(msg) Jul 13 23:15:19.474152 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: Jul 13 23:15:19.476158 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e5b2e7-400c-45da-9dd5-c6fe5cb60f78 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:19.581269 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:19.670635 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-62e5b2e7-400c-45da-9dd5-c6fe5cb60f78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a9434e6932c446b8f3ca24db5526358', 'request_id': 'req-62e5b2e7-400c-45da-9dd5-c6fe5cb60f78'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:19.692253 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-62e5b2e7-400c-45da-9dd5-c6fe5cb60f78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a9434e6932c446b8f3ca24db5526358', 'request_id': 'req-62e5b2e7-400c-45da-9dd5-c6fe5cb60f78'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:19.704732 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-62e5b2e7-400c-45da-9dd5-c6fe5cb60f78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a9434e6932c446b8f3ca24db5526358', 'request_id': 'req-62e5b2e7-400c-45da-9dd5-c6fe5cb60f78'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:19.722884 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-62e5b2e7-400c-45da-9dd5-c6fe5cb60f78 admin admin] The token cache is being invalidate because project 5a9434e6932c446b8f3ca24db5526358 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:15:19.723393 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-62e5b2e7-400c-45da-9dd5-c6fe5cb60f78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:19.724155 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-62e5b2e7-400c-45da-9dd5-c6fe5cb60f78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:19.729078 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 820/1648] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:15:19 2022] DELETE /identity/v3/projects/5a9434e6932c446b8f3ca24db5526358 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:19.731501 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b48ee-6e49-44cf-86e4-bf546d11e6a8 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:19.731501 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b48ee-6e49-44cf-86e4-bf546d11e6a8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:19.731501 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b48ee-6e49-44cf-86e4-bf546d11e6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:19.770746 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-0b0b48ee-6e49-44cf-86e4-bf546d11e6a8 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:19.781917 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0b0b48ee-6e49-44cf-86e4-bf546d11e6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:19.878118 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 829/1649] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:19.885947 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c54275-37a7-403a-9f97-ba806cb377cc None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:19.886103 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c54275-37a7-403a-9f97-ba806cb377cc None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:19.886341 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c54275-37a7-403a-9f97-ba806cb377cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:19.918168 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-14c54275-37a7-403a-9f97-ba806cb377cc None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:19.919638 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-14c54275-37a7-403a-9f97-ba806cb377cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:19.936906 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 821/1650] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:19.937918 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b48ee-6e49-44cf-86e4-bf546d11e6a8 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:19.950205 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b48ee-6e49-44cf-86e4-bf546d11e6a8 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:19.957493 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cfa136-f6fb-4ac8-8350-b4e80c6ed9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:19.958185 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cfa136-f6fb-4ac8-8350-b4e80c6ed9df admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:19.958468 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cfa136-f6fb-4ac8-8350-b4e80c6ed9df admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:19.958744 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cfa136-f6fb-4ac8-8350-b4e80c6ed9df admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:19.959490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cfa136-f6fb-4ac8-8350-b4e80c6ed9df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:15:19.959831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cfa136-f6fb-4ac8-8350-b4e80c6ed9df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:19.965051 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cfa136-f6fb-4ac8-8350-b4e80c6ed9df admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:19.991326 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 830/1651] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:15:19.992747 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c54275-37a7-403a-9f97-ba806cb377cc None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:20.005307 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c54275-37a7-403a-9f97-ba806cb377cc None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:20.016857 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f15175-fca2-4898-a93a-b14db0541e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:20.017701 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f15175-fca2-4898-a93a-b14db0541e22 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:20.018081 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f15175-fca2-4898-a93a-b14db0541e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:20.018444 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f15175-fca2-4898-a93a-b14db0541e22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:20.019423 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f15175-fca2-4898-a93a-b14db0541e22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:20.025642 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f15175-fca2-4898-a93a-b14db0541e22 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:20.105076 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 822/1652] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:15:19 2022] POST /identity/v3/projects => generated 381 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:20.106331 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cfa136-f6fb-4ac8-8350-b4e80c6ed9df admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:20.127514 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cfa136-f6fb-4ac8-8350-b4e80c6ed9df admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:20.257822 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead858ff-ec7b-482a-b798-0cd42cc77144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:20.258519 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead858ff-ec7b-482a-b798-0cd42cc77144 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:20.258804 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead858ff-ec7b-482a-b798-0cd42cc77144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:20.259081 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead858ff-ec7b-482a-b798-0cd42cc77144 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:20.259811 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead858ff-ec7b-482a-b798-0cd42cc77144 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:20.261768 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead858ff-ec7b-482a-b798-0cd42cc77144 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:20.335473 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 831/1653] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:15:20 2022] POST /identity/v3/users => generated 291 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:20.339451 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f15175-fca2-4898-a93a-b14db0541e22 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:20.360016 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f15175-fca2-4898-a93a-b14db0541e22 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:20.388411 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb7e689-0a3c-42d8-b90e-22b184227312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:20.389498 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7e689-0a3c-42d8-b90e-22b184227312 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:20.392436 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7e689-0a3c-42d8-b90e-22b184227312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:20.393052 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7e689-0a3c-42d8-b90e-22b184227312 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:20.394220 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb7e689-0a3c-42d8-b90e-22b184227312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:20.398565 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb7e689-0a3c-42d8-b90e-22b184227312 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:20.424224 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 823/1654] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:15:20.428177 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead858ff-ec7b-482a-b798-0cd42cc77144 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:20.434999 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:20.452961 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead858ff-ec7b-482a-b798-0cd42cc77144 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:20.462510 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8485aa-e799-4fce-8b22-7431d3851ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:20.463196 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8485aa-e799-4fce-8b22-7431d3851ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:20.463478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8485aa-e799-4fce-8b22-7431d3851ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:20.463784 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8485aa-e799-4fce-8b22-7431d3851ea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:20.465071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8485aa-e799-4fce-8b22-7431d3851ea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:20.469207 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8485aa-e799-4fce-8b22-7431d3851ea7 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:20.486210 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 832/1655] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:20.490755 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb7e689-0a3c-42d8-b90e-22b184227312 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:20.507090 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb7e689-0a3c-42d8-b90e-22b184227312 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:20.530657 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd90ff4b-3292-40b2-bd75-424161407df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:20.531960 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90ff4b-3292-40b2-bd75-424161407df3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:20.534080 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90ff4b-3292-40b2-bd75-424161407df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:20.534498 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90ff4b-3292-40b2-bd75-424161407df3 admin admin] PATH_INFO: `/v3/projects/42e47b1e95f64d77ac187a819d8be298/users/338087db4ab9474ca0147aaa200e16fa/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:20.576079 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd90ff4b-3292-40b2-bd75-424161407df3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42e47b1e95f64d77ac187a819d8be298, user_id=338087db4ab9474ca0147aaa200e16fa, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:20.579289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd90ff4b-3292-40b2-bd75-424161407df3 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:20.621936 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 824/1656] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:15:20 2022] PUT /identity/v3/projects/42e47b1e95f64d77ac187a819d8be298/users/338087db4ab9474ca0147aaa200e16fa/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:20.634936 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac92d41-cd46-475f-be4b-f2e9d402173e None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:20.635282 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac92d41-cd46-475f-be4b-f2e9d402173e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:20.635591 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac92d41-cd46-475f-be4b-f2e9d402173e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:20.712156 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-8ac92d41-cd46-475f-be4b-f2e9d402173e None None] MFA Rules not processed for user `338087db4ab9474ca0147aaa200e16fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:20.718594 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8ac92d41-cd46-475f-be4b-f2e9d402173e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:20.903031 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 833/1657] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:20.904428 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd90ff4b-3292-40b2-bd75-424161407df3 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:21.100095 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd90ff4b-3292-40b2-bd75-424161407df3 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:21.111699 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ccbe57-5139-4470-ac1c-a5b230c1f454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:21.112822 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ccbe57-5139-4470-ac1c-a5b230c1f454 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:21.113332 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ccbe57-5139-4470-ac1c-a5b230c1f454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:21.125368 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ccbe57-5139-4470-ac1c-a5b230c1f454 admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:21.126559 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ccbe57-5139-4470-ac1c-a5b230c1f454 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:21.130116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ccbe57-5139-4470-ac1c-a5b230c1f454 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:21.189683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dada6e-c584-4245-a5aa-c4d2c21cd4fe None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:21.190088 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dada6e-c584-4245-a5aa-c4d2c21cd4fe None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:21.190524 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dada6e-c584-4245-a5aa-c4d2c21cd4fe None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:21.192634 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 834/1658] 10.222.0.60 () {54 vars in 921 bytes} [Wed Jul 13 23:15:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:15:21.199780 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 825/1659] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:15:20 2022] POST /identity/v3/projects => generated 381 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:21.201473 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dada6e-c584-4245-a5aa-c4d2c21cd4fe None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:21.280071 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e4c24-3c9e-4ac7-8ee1-65eabb58d207 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:21.280511 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e4c24-3c9e-4ac7-8ee1-65eabb58d207 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:21.280889 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e4c24-3c9e-4ac7-8ee1-65eabb58d207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:21.312662 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-6b6e4c24-3c9e-4ac7-8ee1-65eabb58d207 None None] MFA Rules not processed for user `64824ba84a024930bbc177f8a8d9a046`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:21.313917 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-6b6e4c24-3c9e-4ac7-8ee1-65eabb58d207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:21.367422 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:21.424627 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 826/1660] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Jul 13 23:15:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:21.428007 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dada6e-c584-4245-a5aa-c4d2c21cd4fe None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:21.440134 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f765e-74e7-4c29-9ca0-f3314dbde0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:21.440925 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f765e-74e7-4c29-9ca0-f3314dbde0fb admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:21.441214 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f765e-74e7-4c29-9ca0-f3314dbde0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:21.441492 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f765e-74e7-4c29-9ca0-f3314dbde0fb admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:21.442309 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f765e-74e7-4c29-9ca0-f3314dbde0fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:21.454499 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f765e-74e7-4c29-9ca0-f3314dbde0fb admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:21.462441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1c5de-2790-4730-b9c0-fb18f2700d98 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:21.463016 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1c5de-2790-4730-b9c0-fb18f2700d98 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:21.463845 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1c5de-2790-4730-b9c0-fb18f2700d98 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:21.465845 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 827/1661] 10.222.0.60 () {54 vars in 960 bytes} [Wed Jul 13 23:15:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:15:21.472982 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb321e4-cd99-4275-be49-25cb6e5adad0 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:21.473339 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb321e4-cd99-4275-be49-25cb6e5adad0 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:21.473837 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb321e4-cd99-4275-be49-25cb6e5adad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:21.522803 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 835/1662] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:15:21 2022] POST /identity/v3/users => generated 297 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:21.526081 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f765e-74e7-4c29-9ca0-f3314dbde0fb admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:21.536091 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-5fb321e4-cd99-4275-be49-25cb6e5adad0 None None] MFA Rules not processed for user `64824ba84a024930bbc177f8a8d9a046`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:21.537423 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-5fb321e4-cd99-4275-be49-25cb6e5adad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:21.561168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f765e-74e7-4c29-9ca0-f3314dbde0fb admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:21.587822 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 828/1663] 10.222.0.60 () {58 vars in 1057 bytes} [Wed Jul 13 23:15:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:15:21.589489 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb321e4-cd99-4275-be49-25cb6e5adad0 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:21.595275 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b820f683-f539-4b64-b266-9ea09f0d9ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:21.595275 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820f683-f539-4b64-b266-9ea09f0d9ed2 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:21.595275 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820f683-f539-4b64-b266-9ea09f0d9ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:21.595275 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820f683-f539-4b64-b266-9ea09f0d9ed2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:21.595930 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b820f683-f539-4b64-b266-9ea09f0d9ed2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:21.597503 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b820f683-f539-4b64-b266-9ea09f0d9ed2 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:21.618910 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb321e4-cd99-4275-be49-25cb6e5adad0 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:21.630038 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee46865-df69-4e39-b4cd-ede7d9ee8dda service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64824ba84a024930bbc177f8a8d9a046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:21.632564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee46865-df69-4e39-b4cd-ede7d9ee8dda service placement] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:21.636271 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee46865-df69-4e39-b4cd-ede7d9ee8dda service placement] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:21.636684 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee46865-df69-4e39-b4cd-ede7d9ee8dda service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:21.643200 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 836/1664] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:21.644577 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b820f683-f539-4b64-b266-9ea09f0d9ed2 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:21.663046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee46865-df69-4e39-b4cd-ede7d9ee8dda service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:21.671841 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee46865-df69-4e39-b4cd-ede7d9ee8dda service placement] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:21.679680 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b820f683-f539-4b64-b266-9ea09f0d9ed2 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:21.698850 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c5b5af-84e5-44c4-a219-453daa2f8661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:21.699850 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c5b5af-84e5-44c4-a219-453daa2f8661 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:21.700071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c5b5af-84e5-44c4-a219-453daa2f8661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:21.700345 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c5b5af-84e5-44c4-a219-453daa2f8661 admin admin] PATH_INFO: `/v3/projects/fa424e8207354f4b88788563754ca3b6/users/44c92e64f7ce4d42ab56e90f06e024fa/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:21.720186 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 829/1665] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:15:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:15:21.740791 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c5b5af-84e5-44c4-a219-453daa2f8661 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa424e8207354f4b88788563754ca3b6, user_id=44c92e64f7ce4d42ab56e90f06e024fa, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:21.744286 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c5b5af-84e5-44c4-a219-453daa2f8661 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:21.794317 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 837/1666] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:15:21 2022] PUT /identity/v3/projects/fa424e8207354f4b88788563754ca3b6/users/44c92e64f7ce4d42ab56e90f06e024fa/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:21.799434 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee46865-df69-4e39-b4cd-ede7d9ee8dda service placement] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:21.829239 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee46865-df69-4e39-b4cd-ede7d9ee8dda service placement] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:22.080177 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb8dda-3b74-4b8b-ad1e-93d47b0c2a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:22.081921 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb8dda-3b74-4b8b-ad1e-93d47b0c2a20 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:22.083615 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb8dda-3b74-4b8b-ad1e-93d47b0c2a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:22.090627 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb8dda-3b74-4b8b-ad1e-93d47b0c2a20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:22.091695 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cb8dda-3b74-4b8b-ad1e-93d47b0c2a20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:22.095045 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cb8dda-3b74-4b8b-ad1e-93d47b0c2a20 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:22.123176 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 830/1667] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:22.125126 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c5b5af-84e5-44c4-a219-453daa2f8661 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:22.149342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c5b5af-84e5-44c4-a219-453daa2f8661 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:22.167704 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef05a0-cc9f-447b-892e-e68daa252cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:22.168940 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef05a0-cc9f-447b-892e-e68daa252cd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:22.169384 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef05a0-cc9f-447b-892e-e68daa252cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:22.170080 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef05a0-cc9f-447b-892e-e68daa252cd6 admin admin] PATH_INFO: `/v3/projects/fa424e8207354f4b88788563754ca3b6/users/44c92e64f7ce4d42ab56e90f06e024fa/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:22.174509 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ef05a0-cc9f-447b-892e-e68daa252cd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa424e8207354f4b88788563754ca3b6, user_id=44c92e64f7ce4d42ab56e90f06e024fa, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:22.178375 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ef05a0-cc9f-447b-892e-e68daa252cd6 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:22.209324 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f6257-57da-49f2-af5e-2508e27b3fdb None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:22.209629 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f6257-57da-49f2-af5e-2508e27b3fdb None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:22.209916 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f6257-57da-49f2-af5e-2508e27b3fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:22.211918 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 838/1668] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:15:22 2022] PUT /identity/v3/projects/fa424e8207354f4b88788563754ca3b6/users/44c92e64f7ce4d42ab56e90f06e024fa/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:22.255643 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-748f6257-57da-49f2-af5e-2508e27b3fdb None None] MFA Rules not processed for user `44c92e64f7ce4d42ab56e90f06e024fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:22.262173 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-748f6257-57da-49f2-af5e-2508e27b3fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:22.473699 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 831/1669] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:22 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:22.488367 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef05a0-cc9f-447b-892e-e68daa252cd6 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:22.490889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-06ef05a0-cc9f-447b-892e-e68daa252cd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:22.547251 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:22.605638 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef05a0-cc9f-447b-892e-e68daa252cd6 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:22.614392 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca8769-4a08-4f52-93c3-6b37ef7f1cad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:22.615472 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca8769-4a08-4f52-93c3-6b37ef7f1cad service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:22.616427 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca8769-4a08-4f52-93c3-6b37ef7f1cad service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:22.618369 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca8769-4a08-4f52-93c3-6b37ef7f1cad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:22.636092 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ca8769-4a08-4f52-93c3-6b37ef7f1cad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:22.637935 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ca8769-4a08-4f52-93c3-6b37ef7f1cad service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:22.665979 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 839/1670] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:15:22 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:15:22.779164 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f6257-57da-49f2-af5e-2508e27b3fdb None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:22.796745 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f6257-57da-49f2-af5e-2508e27b3fdb None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:22.811769 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46b6acf-5549-4ba5-9190-5c03794d9615 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:22.813097 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b6acf-5549-4ba5-9190-5c03794d9615 service cinder] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:22.813404 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b6acf-5549-4ba5-9190-5c03794d9615 service cinder] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:22.813834 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46b6acf-5549-4ba5-9190-5c03794d9615 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:22.894904 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:22.981337 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46b6acf-5549-4ba5-9190-5c03794d9615 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:22.982968 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46b6acf-5549-4ba5-9190-5c03794d9615 service cinder] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:23.013635 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 832/1671] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:15:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:15:24.522418 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca8769-4a08-4f52-93c3-6b37ef7f1cad service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:24.524472 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d6ca8769-4a08-4f52-93c3-6b37ef7f1cad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:24.705101 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca8769-4a08-4f52-93c3-6b37ef7f1cad service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:24.721934 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692cc9d-f36b-41ae-8d88-d1a8d655aea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:24.723386 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692cc9d-f36b-41ae-8d88-d1a8d655aea7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:24.724288 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692cc9d-f36b-41ae-8d88-d1a8d655aea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:24.724827 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692cc9d-f36b-41ae-8d88-d1a8d655aea7 admin admin] PATH_INFO: `/v3/users/0fbe204416374bc3b919d14d45d7a653` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:24.727295 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692cc9d-f36b-41ae-8d88-d1a8d655aea7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fbe204416374bc3b919d14d45d7a653)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:24.730275 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692cc9d-f36b-41ae-8d88-d1a8d655aea7 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:24.816404 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3692cc9d-f36b-41ae-8d88-d1a8d655aea7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fbe204416374bc3b919d14d45d7a653'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:24.818020 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3692cc9d-f36b-41ae-8d88-d1a8d655aea7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fbe204416374bc3b919d14d45d7a653'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:24.838989 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3692cc9d-f36b-41ae-8d88-d1a8d655aea7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fbe204416374bc3b919d14d45d7a653'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:24.847601 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3692cc9d-f36b-41ae-8d88-d1a8d655aea7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fbe204416374bc3b919d14d45d7a653'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:24.859335 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3692cc9d-f36b-41ae-8d88-d1a8d655aea7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fbe204416374bc3b919d14d45d7a653'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:24.870555 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 840/1672] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:15:24 2022] DELETE /identity/v3/users/0fbe204416374bc3b919d14d45d7a653 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:26.003886 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ced2a7-66d8-4a4a-ae29-beed5784a195 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:26.004469 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ced2a7-66d8-4a4a-ae29-beed5784a195 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:26.005172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ced2a7-66d8-4a4a-ae29-beed5784a195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:26.075888 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-b3ced2a7-66d8-4a4a-ae29-beed5784a195 None None] MFA Rules not processed for user `646d135cd569428c8eda3577df418963`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:26.079322 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-b3ced2a7-66d8-4a4a-ae29-beed5784a195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:26.101482 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692cc9d-f36b-41ae-8d88-d1a8d655aea7 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:26.103560 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3692cc9d-f36b-41ae-8d88-d1a8d655aea7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:26.285490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:26.324314 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 833/1673] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:26.343362 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692cc9d-f36b-41ae-8d88-d1a8d655aea7 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:26.348603 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc47cb0-ca83-44a9-b975-1cb699e82a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:26.350667 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc47cb0-ca83-44a9-b975-1cb699e82a0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:26.351099 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc47cb0-ca83-44a9-b975-1cb699e82a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:26.351212 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc47cb0-ca83-44a9-b975-1cb699e82a0b admin admin] PATH_INFO: `/v3/projects/a01073a6773c4ca0b3208a004cd15c18` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:26.352592 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc47cb0-ca83-44a9-b975-1cb699e82a0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a01073a6773c4ca0b3208a004cd15c18)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:26.354629 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc47cb0-ca83-44a9-b975-1cb699e82a0b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:26.446720 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362e963-a009-4fe8-8a5a-949a9b0e1142 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:26.447090 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362e963-a009-4fe8-8a5a-949a9b0e1142 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:26.447305 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362e963-a009-4fe8-8a5a-949a9b0e1142 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:26.448765 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 834/1674] 10.222.0.60 () {54 vars in 961 bytes} [Wed Jul 13 23:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:15:26.465262 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f4937-875a-4eb8-80b4-0dbde1839414 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:26.465648 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f4937-875a-4eb8-80b4-0dbde1839414 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:26.465821 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f4937-875a-4eb8-80b4-0dbde1839414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:26.477585 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3fc47cb0-ca83-44a9-b975-1cb699e82a0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a01073a6773c4ca0b3208a004cd15c18', 'request_id': 'req-3fc47cb0-ca83-44a9-b975-1cb699e82a0b'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:26.503218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3fc47cb0-ca83-44a9-b975-1cb699e82a0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a01073a6773c4ca0b3208a004cd15c18', 'request_id': 'req-3fc47cb0-ca83-44a9-b975-1cb699e82a0b'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:26.534430 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3fc47cb0-ca83-44a9-b975-1cb699e82a0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a01073a6773c4ca0b3208a004cd15c18', 'request_id': 'req-3fc47cb0-ca83-44a9-b975-1cb699e82a0b'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:26.587255 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3fc47cb0-ca83-44a9-b975-1cb699e82a0b admin admin] The token cache is being invalidate because project a01073a6773c4ca0b3208a004cd15c18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:15:26.587744 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3fc47cb0-ca83-44a9-b975-1cb699e82a0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:26.589724 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3fc47cb0-ca83-44a9-b975-1cb699e82a0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:26.595032 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 841/1675] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:15:26 2022] DELETE /identity/v3/projects/a01073a6773c4ca0b3208a004cd15c18 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:15:26.598724 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc47cb0-ca83-44a9-b975-1cb699e82a0b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:26.600206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3fc47cb0-ca83-44a9-b975-1cb699e82a0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:26.628422 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-9b0f4937-875a-4eb8-80b4-0dbde1839414 None None] MFA Rules not processed for user `4789384d158b4995bd30e1e37e30f5f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:26.632630 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-9b0f4937-875a-4eb8-80b4-0dbde1839414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:26.733810 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc47cb0-ca83-44a9-b975-1cb699e82a0b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:26.738849 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c6f37-ef51-4ad5-a334-ff9bc1a3746d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:26.739832 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c6f37-ef51-4ad5-a334-ff9bc1a3746d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:26.740270 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c6f37-ef51-4ad5-a334-ff9bc1a3746d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:26.740550 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c6f37-ef51-4ad5-a334-ff9bc1a3746d admin admin] PATH_INFO: `/v3/users/49e2ce186d9b421fbccdcaa67c030c9a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:26.742551 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733c6f37-ef51-4ad5-a334-ff9bc1a3746d admin admin] RBAC: Authorizing `identity:delete_user(user_id=49e2ce186d9b421fbccdcaa67c030c9a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:26.744912 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733c6f37-ef51-4ad5-a334-ff9bc1a3746d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:26.830552 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-733c6f37-ef51-4ad5-a334-ff9bc1a3746d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49e2ce186d9b421fbccdcaa67c030c9a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:26.831272 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-733c6f37-ef51-4ad5-a334-ff9bc1a3746d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49e2ce186d9b421fbccdcaa67c030c9a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:26.860410 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-733c6f37-ef51-4ad5-a334-ff9bc1a3746d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49e2ce186d9b421fbccdcaa67c030c9a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:26.867278 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 835/1676] 10.222.0.60 () {58 vars in 1058 bytes} [Wed Jul 13 23:15:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 403 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:15:26.868965 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-733c6f37-ef51-4ad5-a334-ff9bc1a3746d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49e2ce186d9b421fbccdcaa67c030c9a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:26.874247 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:26.877612 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f4937-875a-4eb8-80b4-0dbde1839414 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:26.890204 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-733c6f37-ef51-4ad5-a334-ff9bc1a3746d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49e2ce186d9b421fbccdcaa67c030c9a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:26.896586 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f4937-875a-4eb8-80b4-0dbde1839414 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:26.901202 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 842/1677] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:15:26 2022] DELETE /identity/v3/users/49e2ce186d9b421fbccdcaa67c030c9a => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:26.917125 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12c1006-e868-453b-8446-1430f8064886 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:26.917995 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c1006-e868-453b-8446-1430f8064886 service nova] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:26.918287 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c1006-e868-453b-8446-1430f8064886 service nova] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:26.923604 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c1006-e868-453b-8446-1430f8064886 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:26.925941 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-b12c1006-e868-453b-8446-1430f8064886 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:27.076777 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:27.142183 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12c1006-e868-453b-8446-1430f8064886 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:27.143939 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12c1006-e868-453b-8446-1430f8064886 service nova] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:27.175138 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 836/1678] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:15:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:15:27.479405 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa7838-e45f-4f62-9c6e-6fc776cfd80c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:27.479933 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa7838-e45f-4f62-9c6e-6fc776cfd80c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:27.480058 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa7838-e45f-4f62-9c6e-6fc776cfd80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:27.561548 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-9efa7838-e45f-4f62-9c6e-6fc776cfd80c None None] MFA Rules not processed for user `7ade2669af504e91936edb12c5d919a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:27.562801 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9efa7838-e45f-4f62-9c6e-6fc776cfd80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:27.738514 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 843/1679] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:27.750181 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58065653-9f58-4666-b4d6-811276a8f979 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:27.751019 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58065653-9f58-4666-b4d6-811276a8f979 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:27.751019 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58065653-9f58-4666-b4d6-811276a8f979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:27.795071 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-58065653-9f58-4666-b4d6-811276a8f979 None None] MFA Rules not processed for user `7ade2669af504e91936edb12c5d919a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:27.801227 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-58065653-9f58-4666-b4d6-811276a8f979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:27.840073 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 837/1680] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:27.882803 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf0077-2d64-4cff-8d3f-3a58533cff48 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:27.883419 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf0077-2d64-4cff-8d3f-3a58533cff48 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:27.883804 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf0077-2d64-4cff-8d3f-3a58533cff48 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:27.885977 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 844/1681] 10.222.0.60 () {54 vars in 961 bytes} [Wed Jul 13 23:15:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:15:27.926552 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb5684b-716a-458c-ac5d-3f2a10c5200d None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:27.927115 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb5684b-716a-458c-ac5d-3f2a10c5200d None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:27.927405 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb5684b-716a-458c-ac5d-3f2a10c5200d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:28.009943 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-acb5684b-716a-458c-ac5d-3f2a10c5200d None None] MFA Rules not processed for user `4789384d158b4995bd30e1e37e30f5f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:28.011200 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-acb5684b-716a-458c-ac5d-3f2a10c5200d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:28.286652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 838/1682] 10.222.0.60 () {58 vars in 1058 bytes} [Wed Jul 13 23:15:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:28.298900 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcf0077-2d64-4cff-8d3f-3a58533cff48 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:28.328113 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcf0077-2d64-4cff-8d3f-3a58533cff48 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:28.350275 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc1a56-52d3-4532-ba10-c8fa3bbf2291 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:28.350967 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc1a56-52d3-4532-ba10-c8fa3bbf2291 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:28.351256 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc1a56-52d3-4532-ba10-c8fa3bbf2291 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:28.351557 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc1a56-52d3-4532-ba10-c8fa3bbf2291 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:28.386413 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccc1a56-52d3-4532-ba10-c8fa3bbf2291 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:28.395826 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccc1a56-52d3-4532-ba10-c8fa3bbf2291 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:28.446042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 845/1683] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:15:28 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:15:28.704596 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb5684b-716a-458c-ac5d-3f2a10c5200d None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:28.706634 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-acb5684b-716a-458c-ac5d-3f2a10c5200d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:28.905718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb5684b-716a-458c-ac5d-3f2a10c5200d None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:28.915401 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fd2000-e13f-4aaf-8d6e-0309200b63e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:28.917535 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd2000-e13f-4aaf-8d6e-0309200b63e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:28.918994 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd2000-e13f-4aaf-8d6e-0309200b63e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:28.918994 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd2000-e13f-4aaf-8d6e-0309200b63e0 admin admin] PATH_INFO: `/v3/projects/f7678a8dfe21460184a8ab7206e3244d` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:28.923581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fd2000-e13f-4aaf-8d6e-0309200b63e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7678a8dfe21460184a8ab7206e3244d)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:28.929725 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fd2000-e13f-4aaf-8d6e-0309200b63e0 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:29.034050 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e73fd77-85ab-4597-a15a-7a29d9808547 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:29.034590 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e73fd77-85ab-4597-a15a-7a29d9808547 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:29.034696 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e73fd77-85ab-4597-a15a-7a29d9808547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:29.068459 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-45fd2000-e13f-4aaf-8d6e-0309200b63e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7678a8dfe21460184a8ab7206e3244d', 'request_id': 'req-45fd2000-e13f-4aaf-8d6e-0309200b63e0'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:29.084348 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-45fd2000-e13f-4aaf-8d6e-0309200b63e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7678a8dfe21460184a8ab7206e3244d', 'request_id': 'req-45fd2000-e13f-4aaf-8d6e-0309200b63e0'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:29.094441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:29.097463 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-8e73fd77-85ab-4597-a15a-7a29d9808547 None None] MFA Rules not processed for user `5e052654361f4580ab36a347610f63c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:29.099526 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8e73fd77-85ab-4597-a15a-7a29d9808547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:29.104935 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-45fd2000-e13f-4aaf-8d6e-0309200b63e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7678a8dfe21460184a8ab7206e3244d', 'request_id': 'req-45fd2000-e13f-4aaf-8d6e-0309200b63e0'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:29.134244 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-45fd2000-e13f-4aaf-8d6e-0309200b63e0 admin admin] The token cache is being invalidate because project f7678a8dfe21460184a8ab7206e3244d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:15:29.135446 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-45fd2000-e13f-4aaf-8d6e-0309200b63e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:29.137378 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-45fd2000-e13f-4aaf-8d6e-0309200b63e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:29.146380 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 839/1684] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:15:28 2022] DELETE /identity/v3/projects/f7678a8dfe21460184a8ab7206e3244d => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:29.153415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea2a468-13ba-4c56-8713-bb259907498e None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:29.155016 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea2a468-13ba-4c56-8713-bb259907498e None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:29.155695 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea2a468-13ba-4c56-8713-bb259907498e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:29.211594 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-7ea2a468-13ba-4c56-8713-bb259907498e None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:29.212842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-7ea2a468-13ba-4c56-8713-bb259907498e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:29.303460 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 846/1685] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:29.305234 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590dd86a-db3f-4431-8182-57c3611703ca None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:29.309707 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590dd86a-db3f-4431-8182-57c3611703ca None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:29.310173 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590dd86a-db3f-4431-8182-57c3611703ca None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:29.314323 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 847/1686] 10.222.0.60 () {54 vars in 912 bytes} [Wed Jul 13 23:15:29 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:15:29.315659 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590dd86a-db3f-4431-8182-57c3611703ca None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:29.319793 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-590dd86a-db3f-4431-8182-57c3611703ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:29.347277 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 840/1687] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:29.349416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231dd8e3-f188-4049-bbd9-307353bcb42e None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:29.349890 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231dd8e3-f188-4049-bbd9-307353bcb42e None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:29.350306 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231dd8e3-f188-4049-bbd9-307353bcb42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:29.413991 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-231dd8e3-f188-4049-bbd9-307353bcb42e None None] MFA Rules not processed for user `4789384d158b4995bd30e1e37e30f5f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:29.415435 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-231dd8e3-f188-4049-bbd9-307353bcb42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:29.515332 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:29.576913 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 841/1688] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Jul 13 23:15:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:29.578629 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604ab9c-9bde-4240-8014-2ecd1320ab6b None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:29.579114 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604ab9c-9bde-4240-8014-2ecd1320ab6b None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:29.579470 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604ab9c-9bde-4240-8014-2ecd1320ab6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:29.603571 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-5604ab9c-9bde-4240-8014-2ecd1320ab6b None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:29.604928 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-5604ab9c-9bde-4240-8014-2ecd1320ab6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:29.613534 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590dd86a-db3f-4431-8182-57c3611703ca None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:29.618701 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 842/1689] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:29.619611 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604ab9c-9bde-4240-8014-2ecd1320ab6b None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:29.621486 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-5604ab9c-9bde-4240-8014-2ecd1320ab6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:29.643393 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20310044-ef21-4857-a0e9-aa9a5bb309ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:29.647882 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20310044-ef21-4857-a0e9-aa9a5bb309ad service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:29.648175 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20310044-ef21-4857-a0e9-aa9a5bb309ad service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:29.650931 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20310044-ef21-4857-a0e9-aa9a5bb309ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:29.686328 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20310044-ef21-4857-a0e9-aa9a5bb309ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:29.692716 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20310044-ef21-4857-a0e9-aa9a5bb309ad service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:29.730832 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 848/1690] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:15:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:15:29.753928 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604ab9c-9bde-4240-8014-2ecd1320ab6b None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:29.758917 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84209a20-421e-4d08-8526-21442c609f9d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ce9e6497fdb4068b23aa3f7b2c19536', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:29.759751 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84209a20-421e-4d08-8526-21442c609f9d service glance] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:29.760163 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84209a20-421e-4d08-8526-21442c609f9d service glance] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:29.760548 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84209a20-421e-4d08-8526-21442c609f9d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:29.762392 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20310044-ef21-4857-a0e9-aa9a5bb309ad service nova] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:29.794820 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:29.809114 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84209a20-421e-4d08-8526-21442c609f9d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:29.812525 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20310044-ef21-4857-a0e9-aa9a5bb309ad service nova] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:29.817048 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84209a20-421e-4d08-8526-21442c609f9d service glance] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:29.828942 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3cfb79-1d35-4eb1-9fe1-41092d32783b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:29.829696 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3cfb79-1d35-4eb1-9fe1-41092d32783b admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:29.829913 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3cfb79-1d35-4eb1-9fe1-41092d32783b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:29.830187 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3cfb79-1d35-4eb1-9fe1-41092d32783b admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:29.834667 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3cfb79-1d35-4eb1-9fe1-41092d32783b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:15:29.835017 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3cfb79-1d35-4eb1-9fe1-41092d32783b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:29.836987 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3cfb79-1d35-4eb1-9fe1-41092d32783b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:29.875908 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 843/1691] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:15:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:15:29.883600 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 849/1692] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:15:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:15:29.887953 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84209a20-421e-4d08-8526-21442c609f9d service glance] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:29.913478 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84209a20-421e-4d08-8526-21442c609f9d service glance] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:29.926856 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2d96d-3e27-4bdd-aa07-ad97abfc453c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:29.928216 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2d96d-3e27-4bdd-aa07-ad97abfc453c admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:29.928444 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2d96d-3e27-4bdd-aa07-ad97abfc453c admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:29.928677 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2d96d-3e27-4bdd-aa07-ad97abfc453c admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:29.929386 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c2d96d-3e27-4bdd-aa07-ad97abfc453c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:29.931327 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c2d96d-3e27-4bdd-aa07-ad97abfc453c admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:30.063777 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 844/1693] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:15:29 2022] POST /identity/v3/projects => generated 365 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:15:30.068073 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3cfb79-1d35-4eb1-9fe1-41092d32783b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:30.085411 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3cfb79-1d35-4eb1-9fe1-41092d32783b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:30.262492 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0f33f-4cbd-4595-b0ad-5060957a5dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:30.263200 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0f33f-4cbd-4595-b0ad-5060957a5dea admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:30.263482 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0f33f-4cbd-4595-b0ad-5060957a5dea admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:30.263785 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0f33f-4cbd-4595-b0ad-5060957a5dea admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:30.265056 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d0f33f-4cbd-4595-b0ad-5060957a5dea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:30.267301 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d0f33f-4cbd-4595-b0ad-5060957a5dea admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:30.347793 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 850/1694] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:15:30 2022] POST /identity/v3/users => generated 283 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:30.351395 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2d96d-3e27-4bdd-aa07-ad97abfc453c admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:30.367243 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2d96d-3e27-4bdd-aa07-ad97abfc453c admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:30.380168 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63415c7f-ca7a-4752-af2f-815ab8ddd3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:30.381128 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63415c7f-ca7a-4752-af2f-815ab8ddd3d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:30.381562 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63415c7f-ca7a-4752-af2f-815ab8ddd3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:30.383326 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63415c7f-ca7a-4752-af2f-815ab8ddd3d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:30.384823 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63415c7f-ca7a-4752-af2f-815ab8ddd3d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:30.387721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63415c7f-ca7a-4752-af2f-815ab8ddd3d8 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:30.406134 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 845/1695] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:30.409656 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0f33f-4cbd-4595-b0ad-5060957a5dea admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:30.427924 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0f33f-4cbd-4595-b0ad-5060957a5dea admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:30.438185 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70634d4-6919-4c15-848f-baa9acf6963d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:30.440300 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70634d4-6919-4c15-848f-baa9acf6963d admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:30.440935 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70634d4-6919-4c15-848f-baa9acf6963d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:30.442055 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70634d4-6919-4c15-848f-baa9acf6963d admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:30.444118 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70634d4-6919-4c15-848f-baa9acf6963d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:30.449858 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70634d4-6919-4c15-848f-baa9acf6963d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:30.487858 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 851/1696] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:30.489595 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63415c7f-ca7a-4752-af2f-815ab8ddd3d8 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:30.502579 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63415c7f-ca7a-4752-af2f-815ab8ddd3d8 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:30.513843 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e712ac53-523e-45dc-918c-7a6fd2a65cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:30.515197 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e712ac53-523e-45dc-918c-7a6fd2a65cb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:30.515675 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e712ac53-523e-45dc-918c-7a6fd2a65cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:30.516166 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e712ac53-523e-45dc-918c-7a6fd2a65cb0 admin admin] PATH_INFO: `/v3/projects/c576fa58163043fa8f57e7d334ce328b/users/b98e5cf826bc4a49a9d3dc098b486f59/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:30.569684 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e712ac53-523e-45dc-918c-7a6fd2a65cb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c576fa58163043fa8f57e7d334ce328b, user_id=b98e5cf826bc4a49a9d3dc098b486f59, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:30.576179 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e712ac53-523e-45dc-918c-7a6fd2a65cb0 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:30.604073 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 846/1697] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:15:30 2022] PUT /identity/v3/projects/c576fa58163043fa8f57e7d334ce328b/users/b98e5cf826bc4a49a9d3dc098b486f59/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:30.616672 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95b514-1eaf-45af-8bf9-5953f22c5813 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:30.616958 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95b514-1eaf-45af-8bf9-5953f22c5813 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:30.617362 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95b514-1eaf-45af-8bf9-5953f22c5813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:30.677291 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-4c95b514-1eaf-45af-8bf9-5953f22c5813 None None] MFA Rules not processed for user `b98e5cf826bc4a49a9d3dc098b486f59`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:30.679347 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4c95b514-1eaf-45af-8bf9-5953f22c5813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:30.771607 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6f1e9-b6aa-4cfa-95cc-9d348c1cde8c None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:30.771897 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6f1e9-b6aa-4cfa-95cc-9d348c1cde8c None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:30.772181 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6f1e9-b6aa-4cfa-95cc-9d348c1cde8c None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:30.777039 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 847/1698] 10.222.0.60 () {54 vars in 912 bytes} [Wed Jul 13 23:15:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:15:30.792662 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8238ad9-81bd-4264-821e-e1c567702808 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:30.793222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8238ad9-81bd-4264-821e-e1c567702808 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:30.793702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8238ad9-81bd-4264-821e-e1c567702808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:30.895219 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-e8238ad9-81bd-4264-821e-e1c567702808 None None] MFA Rules not processed for user `2d670df2aff5478ebf61f55358d554de`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:30.899025 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-e8238ad9-81bd-4264-821e-e1c567702808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:30.906520 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 852/1699] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:30 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:30.907583 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c95b514-1eaf-45af-8bf9-5953f22c5813 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:31.005002 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 848/1700] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Jul 13 23:15:30 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:31.033411 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:31.077743 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c95b514-1eaf-45af-8bf9-5953f22c5813 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:31.091070 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a9dc5-3cc8-4942-912b-93c64888c00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:31.091788 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a9dc5-3cc8-4942-912b-93c64888c00b admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:31.092608 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a9dc5-3cc8-4942-912b-93c64888c00b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:31.092886 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a9dc5-3cc8-4942-912b-93c64888c00b admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:31.093599 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a9dc5-3cc8-4942-912b-93c64888c00b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:31.097386 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a9dc5-3cc8-4942-912b-93c64888c00b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:31.161413 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 853/1701] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:15:30 2022] POST /identity/v3/projects => generated 363 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:31.163024 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8238ad9-81bd-4264-821e-e1c567702808 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:31.185693 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8238ad9-81bd-4264-821e-e1c567702808 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:31.273624 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a9dc5-3cc8-4942-912b-93c64888c00b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:31.275487 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-fc9a9dc5-3cc8-4942-912b-93c64888c00b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:31.336700 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad390af-c1d3-4859-a410-e44f83dda0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:31.338604 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad390af-c1d3-4859-a410-e44f83dda0f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:31.338888 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad390af-c1d3-4859-a410-e44f83dda0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:31.343566 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad390af-c1d3-4859-a410-e44f83dda0f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:31.343566 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad390af-c1d3-4859-a410-e44f83dda0f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:31.350549 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad390af-c1d3-4859-a410-e44f83dda0f5 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:31.387910 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 849/1702] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:15:31 2022] POST /identity/v3/users => generated 288 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:31.391926 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad390af-c1d3-4859-a410-e44f83dda0f5 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:31.411948 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a9dc5-3cc8-4942-912b-93c64888c00b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:31.416547 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad390af-c1d3-4859-a410-e44f83dda0f5 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:31.417070 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f5dfd-5806-4960-a036-02fd5e0079b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:31.417738 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f5dfd-5806-4960-a036-02fd5e0079b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:31.418022 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f5dfd-5806-4960-a036-02fd5e0079b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:31.418704 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f5dfd-5806-4960-a036-02fd5e0079b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:31.423764 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c7be22-5cbb-4084-b222-311e4f9431ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:31.424673 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c7be22-5cbb-4084-b222-311e4f9431ea admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:31.425043 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c7be22-5cbb-4084-b222-311e4f9431ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:31.425419 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c7be22-5cbb-4084-b222-311e4f9431ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:31.427133 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c7be22-5cbb-4084-b222-311e4f9431ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:31.430558 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c7be22-5cbb-4084-b222-311e4f9431ea admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:31.439962 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0f5dfd-5806-4960-a036-02fd5e0079b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:31.444643 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0f5dfd-5806-4960-a036-02fd5e0079b8 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:31.446127 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 850/1703] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:31.452858 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c7be22-5cbb-4084-b222-311e4f9431ea admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:31.465101 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 854/1704] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:15:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:15:31.467995 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c7be22-5cbb-4084-b222-311e4f9431ea admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:31.473319 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e6f8fc-bac9-4253-b269-997bb8573069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:31.474416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e6f8fc-bac9-4253-b269-997bb8573069 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:31.474741 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e6f8fc-bac9-4253-b269-997bb8573069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:31.474973 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e6f8fc-bac9-4253-b269-997bb8573069 admin admin] PATH_INFO: `/v3/projects/a4eb70d5598a4ff6962a8f0170a8b3b8/users/c901e8d1df744ce2af90f17d3c7a2621/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:31.527222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:31.532923 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e6f8fc-bac9-4253-b269-997bb8573069 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4eb70d5598a4ff6962a8f0170a8b3b8, user_id=c901e8d1df744ce2af90f17d3c7a2621, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:31.538689 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e6f8fc-bac9-4253-b269-997bb8573069 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:31.569816 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 851/1705] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:15:31 2022] PUT /identity/v3/projects/a4eb70d5598a4ff6962a8f0170a8b3b8/users/c901e8d1df744ce2af90f17d3c7a2621/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:31.574152 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f5dfd-5806-4960-a036-02fd5e0079b8 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:31.590152 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f5dfd-5806-4960-a036-02fd5e0079b8 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:31.727231 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7353fce0-025a-4958-a034-abe9e745faa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:31.728038 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7353fce0-025a-4958-a034-abe9e745faa9 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:31.728387 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7353fce0-025a-4958-a034-abe9e745faa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:31.728734 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7353fce0-025a-4958-a034-abe9e745faa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:31.729548 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7353fce0-025a-4958-a034-abe9e745faa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:31.731240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7353fce0-025a-4958-a034-abe9e745faa9 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:31.766698 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 855/1706] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:31.772956 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e6f8fc-bac9-4253-b269-997bb8573069 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:31.785942 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e6f8fc-bac9-4253-b269-997bb8573069 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:31.809641 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0415f6fe-fa60-4bea-8f04-76db91b8bb21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:31.810572 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415f6fe-fa60-4bea-8f04-76db91b8bb21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:31.811869 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415f6fe-fa60-4bea-8f04-76db91b8bb21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:31.812719 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0415f6fe-fa60-4bea-8f04-76db91b8bb21 admin admin] PATH_INFO: `/v3/projects/a4eb70d5598a4ff6962a8f0170a8b3b8/users/c901e8d1df744ce2af90f17d3c7a2621/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:31.818732 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0415f6fe-fa60-4bea-8f04-76db91b8bb21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4eb70d5598a4ff6962a8f0170a8b3b8, user_id=c901e8d1df744ce2af90f17d3c7a2621, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:31.823785 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0415f6fe-fa60-4bea-8f04-76db91b8bb21 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:31.868752 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df900d1-230e-49b1-99d5-7642d9c06ca5 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:31.869041 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df900d1-230e-49b1-99d5-7642d9c06ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:31.869396 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df900d1-230e-49b1-99d5-7642d9c06ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:31.871427 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 852/1707] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:15:31 2022] PUT /identity/v3/projects/a4eb70d5598a4ff6962a8f0170a8b3b8/users/c901e8d1df744ce2af90f17d3c7a2621/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:31.918625 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-4df900d1-230e-49b1-99d5-7642d9c06ca5 None None] MFA Rules not processed for user `c901e8d1df744ce2af90f17d3c7a2621`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:31.920579 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4df900d1-230e-49b1-99d5-7642d9c06ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:32.040894 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:32.114659 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 856/1708] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:31 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:32.123828 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0415f6fe-fa60-4bea-8f04-76db91b8bb21 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:32.125843 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-0415f6fe-fa60-4bea-8f04-76db91b8bb21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:32.168068 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da415346-eda9-4dc6-b2bb-b7d3da43b4cc None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:32.168931 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da415346-eda9-4dc6-b2bb-b7d3da43b4cc None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:32.168931 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da415346-eda9-4dc6-b2bb-b7d3da43b4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:32.247501 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0415f6fe-fa60-4bea-8f04-76db91b8bb21 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:32.254762 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c5ef42-ebf8-4503-aee3-36ac1b0e91bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:32.255986 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5ef42-ebf8-4503-aee3-36ac1b0e91bf service cinder] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:32.257229 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5ef42-ebf8-4503-aee3-36ac1b0e91bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:32.257883 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5ef42-ebf8-4503-aee3-36ac1b0e91bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:32.274660 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-da415346-eda9-4dc6-b2bb-b7d3da43b4cc None None] MFA Rules not processed for user `e2e2bd6cf0a44d529ef9b2678c467aba`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:32.281086 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-da415346-eda9-4dc6-b2bb-b7d3da43b4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:32.300809 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c5ef42-ebf8-4503-aee3-36ac1b0e91bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:32.308745 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c5ef42-ebf8-4503-aee3-36ac1b0e91bf service cinder] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:32.372635 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 853/1709] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:15:32 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:15:32.554659 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 857/1710] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:32.565440 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceeb837-d9b7-4b6c-8dac-a3df27315292 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:32.565922 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceeb837-d9b7-4b6c-8dac-a3df27315292 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:32.566018 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceeb837-d9b7-4b6c-8dac-a3df27315292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:32.606871 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-fceeb837-d9b7-4b6c-8dac-a3df27315292 None None] MFA Rules not processed for user `e2e2bd6cf0a44d529ef9b2678c467aba`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:32.609417 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-fceeb837-d9b7-4b6c-8dac-a3df27315292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:32.613083 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b2015-f728-480b-b49c-848b2e32b2c5 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:32.613695 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b2015-f728-480b-b49c-848b2e32b2c5 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:32.613695 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b2015-f728-480b-b49c-848b2e32b2c5 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:32.615088 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 858/1711] 10.222.0.60 () {54 vars in 912 bytes} [Wed Jul 13 23:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:15:32.676042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0791644-11e1-4344-80e9-828e14f99fe1 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:32.676330 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0791644-11e1-4344-80e9-828e14f99fe1 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:32.676610 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0791644-11e1-4344-80e9-828e14f99fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:32.683677 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 854/1712] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:32.693606 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceeb837-d9b7-4b6c-8dac-a3df27315292 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:32.696280 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-fceeb837-d9b7-4b6c-8dac-a3df27315292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:32.749285 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-b0791644-11e1-4344-80e9-828e14f99fe1 None None] MFA Rules not processed for user `4789384d158b4995bd30e1e37e30f5f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:32.754296 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b0791644-11e1-4344-80e9-828e14f99fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:32.969730 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 859/1713] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Jul 13 23:15:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:32.970634 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0791644-11e1-4344-80e9-828e14f99fe1 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:32.972273 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b0791644-11e1-4344-80e9-828e14f99fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:33.003719 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceeb837-d9b7-4b6c-8dac-a3df27315292 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:33.018727 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6072a82e-0759-4bc9-9890-c74b1901cb50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:33.020421 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072a82e-0759-4bc9-9890-c74b1901cb50 service nova] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:33.022623 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072a82e-0759-4bc9-9890-c74b1901cb50 service nova] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:33.023194 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072a82e-0759-4bc9-9890-c74b1901cb50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:33.047135 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6072a82e-0759-4bc9-9890-c74b1901cb50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:33.048859 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6072a82e-0759-4bc9-9890-c74b1901cb50 service nova] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:33.107113 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 855/1714] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:15:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:15:33.108170 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6072a82e-0759-4bc9-9890-c74b1901cb50 service nova] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:33.171214 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0791644-11e1-4344-80e9-828e14f99fe1 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:33.189821 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908bea79-9544-4023-a9d4-2dd2e0ad4b2e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64824ba84a024930bbc177f8a8d9a046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:33.190738 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908bea79-9544-4023-a9d4-2dd2e0ad4b2e service placement] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:33.191168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908bea79-9544-4023-a9d4-2dd2e0ad4b2e service placement] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:33.191671 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908bea79-9544-4023-a9d4-2dd2e0ad4b2e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:33.197226 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-908bea79-9544-4023-a9d4-2dd2e0ad4b2e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:33.236827 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908bea79-9544-4023-a9d4-2dd2e0ad4b2e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:33.238455 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908bea79-9544-4023-a9d4-2dd2e0ad4b2e service placement] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:33.277232 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:33.280873 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6072a82e-0759-4bc9-9890-c74b1901cb50 service nova] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:33.287167 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 860/1715] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:15:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:15:33.288844 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d8d18-1e6a-4e29-9655-9d38320476c3 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:33.290387 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d17970d-4c81-45a0-9f72-e8c10fc9e853 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:33.290873 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d8d18-1e6a-4e29-9655-9d38320476c3 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:33.291127 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17970d-4c81-45a0-9f72-e8c10fc9e853 service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:33.291416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17970d-4c81-45a0-9f72-e8c10fc9e853 service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:33.291675 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d8d18-1e6a-4e29-9655-9d38320476c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:33.291797 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17970d-4c81-45a0-9f72-e8c10fc9e853 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:33.322762 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d17970d-4c81-45a0-9f72-e8c10fc9e853 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:33.334133 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-a52d8d18-1e6a-4e29-9655-9d38320476c3 None None] MFA Rules not processed for user `646d135cd569428c8eda3577df418963`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:33.339675 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d17970d-4c81-45a0-9f72-e8c10fc9e853 service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:33.341235 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a52d8d18-1e6a-4e29-9655-9d38320476c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:33.392398 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:33.426050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 856/1716] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:15:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:15:33.444318 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a04ad1-3f61-4a6e-87a3-fe65ffcdecf4 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:33.444693 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a04ad1-3f61-4a6e-87a3-fe65ffcdecf4 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:33.444978 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a04ad1-3f61-4a6e-87a3-fe65ffcdecf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:33.519956 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-65a04ad1-3f61-4a6e-87a3-fe65ffcdecf4 None None] MFA Rules not processed for user `75652cd3bfa04c7ebae2cbea77cb5fd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:33.524234 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-65a04ad1-3f61-4a6e-87a3-fe65ffcdecf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:33.573053 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 861/1717] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:33.791975 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 857/1718] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:15:34.147408 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d8d18-1e6a-4e29-9655-9d38320476c3 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:34.422379 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d8d18-1e6a-4e29-9655-9d38320476c3 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:34.441703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd7972-ba21-4471-8893-911f81d91204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:34.447157 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd7972-ba21-4471-8893-911f81d91204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:34.448608 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd7972-ba21-4471-8893-911f81d91204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:34.450382 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd7972-ba21-4471-8893-911f81d91204 admin admin] PATH_INFO: `/v3/users/b98e5cf826bc4a49a9d3dc098b486f59` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:34.455931 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd7972-ba21-4471-8893-911f81d91204 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b98e5cf826bc4a49a9d3dc098b486f59)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:34.470712 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd7972-ba21-4471-8893-911f81d91204 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:34.671810 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-28fd7972-ba21-4471-8893-911f81d91204 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b98e5cf826bc4a49a9d3dc098b486f59'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:34.672696 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-28fd7972-ba21-4471-8893-911f81d91204 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b98e5cf826bc4a49a9d3dc098b486f59'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:34.710744 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-28fd7972-ba21-4471-8893-911f81d91204 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b98e5cf826bc4a49a9d3dc098b486f59'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:34.730561 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-28fd7972-ba21-4471-8893-911f81d91204 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b98e5cf826bc4a49a9d3dc098b486f59'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:34.755596 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-28fd7972-ba21-4471-8893-911f81d91204 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b98e5cf826bc4a49a9d3dc098b486f59'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:34.767040 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 862/1719] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:15:34 2022] DELETE /identity/v3/users/b98e5cf826bc4a49a9d3dc098b486f59 => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:34.768820 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60395f6c-488d-457e-b052-a54dc9c0c9dc None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:34.769114 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60395f6c-488d-457e-b052-a54dc9c0c9dc None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:34.769401 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60395f6c-488d-457e-b052-a54dc9c0c9dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:34.799908 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-60395f6c-488d-457e-b052-a54dc9c0c9dc None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:34.801795 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-60395f6c-488d-457e-b052-a54dc9c0c9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:35.009056 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 858/1720] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:35.017314 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd7972-ba21-4471-8893-911f81d91204 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:35.020790 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-28fd7972-ba21-4471-8893-911f81d91204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:35.079263 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:35.159762 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd7972-ba21-4471-8893-911f81d91204 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:35.172568 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add9398-0797-4bbd-a45a-c5ccfba205f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:35.173179 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add9398-0797-4bbd-a45a-c5ccfba205f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:35.174593 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add9398-0797-4bbd-a45a-c5ccfba205f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:35.174880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add9398-0797-4bbd-a45a-c5ccfba205f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:35.195009 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3add9398-0797-4bbd-a45a-c5ccfba205f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:35.197483 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3add9398-0797-4bbd-a45a-c5ccfba205f7 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:35.212547 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 863/1721] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:15:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:15:35.493438 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2324ff-2857-448d-802e-3e69ca180fc3 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:35.493849 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2324ff-2857-448d-802e-3e69ca180fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:35.494137 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2324ff-2857-448d-802e-3e69ca180fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:35.522861 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-1d2324ff-2857-448d-802e-3e69ca180fc3 None None] MFA Rules not processed for user `2d670df2aff5478ebf61f55358d554de`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:35.525012 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-1d2324ff-2857-448d-802e-3e69ca180fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:35.542905 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 859/1722] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Jul 13 23:15:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:35.570383 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add9398-0797-4bbd-a45a-c5ccfba205f7 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:35.572516 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3add9398-0797-4bbd-a45a-c5ccfba205f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:35.610873 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add9398-0797-4bbd-a45a-c5ccfba205f7 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:35.623786 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac9f8f3-adce-4a23-bdda-0be23ed44993 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:35.624623 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9f8f3-adce-4a23-bdda-0be23ed44993 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:35.624980 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9f8f3-adce-4a23-bdda-0be23ed44993 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:35.625329 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9f8f3-adce-4a23-bdda-0be23ed44993 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:35.649056 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac9f8f3-adce-4a23-bdda-0be23ed44993 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:35.654922 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac9f8f3-adce-4a23-bdda-0be23ed44993 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:35.676446 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 864/1723] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:15:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:15:36.023641 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2324ff-2857-448d-802e-3e69ca180fc3 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:36.025771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-1d2324ff-2857-448d-802e-3e69ca180fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:36.053893 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2324ff-2857-448d-802e-3e69ca180fc3 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:36.078709 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc2dbed-f944-4ff8-bf0c-0cfb005b8482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:36.080329 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2dbed-f944-4ff8-bf0c-0cfb005b8482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:36.080825 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2dbed-f944-4ff8-bf0c-0cfb005b8482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:36.082080 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2dbed-f944-4ff8-bf0c-0cfb005b8482 admin admin] PATH_INFO: `/v3/projects/c576fa58163043fa8f57e7d334ce328b` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:36.089227 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc2dbed-f944-4ff8-bf0c-0cfb005b8482 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c576fa58163043fa8f57e7d334ce328b)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:36.098614 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc2dbed-f944-4ff8-bf0c-0cfb005b8482 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:36.283747 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-efc2dbed-f944-4ff8-bf0c-0cfb005b8482 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c576fa58163043fa8f57e7d334ce328b', 'request_id': 'req-efc2dbed-f944-4ff8-bf0c-0cfb005b8482'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:36.308211 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-efc2dbed-f944-4ff8-bf0c-0cfb005b8482 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c576fa58163043fa8f57e7d334ce328b', 'request_id': 'req-efc2dbed-f944-4ff8-bf0c-0cfb005b8482'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:36.345637 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-efc2dbed-f944-4ff8-bf0c-0cfb005b8482 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c576fa58163043fa8f57e7d334ce328b', 'request_id': 'req-efc2dbed-f944-4ff8-bf0c-0cfb005b8482'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:36.384288 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-efc2dbed-f944-4ff8-bf0c-0cfb005b8482 admin admin] The token cache is being invalidate because project c576fa58163043fa8f57e7d334ce328b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:15:36.384594 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-efc2dbed-f944-4ff8-bf0c-0cfb005b8482 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:36.386112 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-efc2dbed-f944-4ff8-bf0c-0cfb005b8482 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:36.393949 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 860/1724] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:15:36 2022] DELETE /identity/v3/projects/c576fa58163043fa8f57e7d334ce328b => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:36.397854 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac9f8f3-adce-4a23-bdda-0be23ed44993 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:36.421124 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac9f8f3-adce-4a23-bdda-0be23ed44993 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:36.782977 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86853bf7-617b-46c9-9741-f479c86f55af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:36.784996 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86853bf7-617b-46c9-9741-f479c86f55af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:36.785358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86853bf7-617b-46c9-9741-f479c86f55af admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:36.785675 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86853bf7-617b-46c9-9741-f479c86f55af admin admin] PATH_INFO: `/v3/users/c901e8d1df744ce2af90f17d3c7a2621` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:36.787520 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86853bf7-617b-46c9-9741-f479c86f55af admin admin] RBAC: Authorizing `identity:delete_user(user_id=c901e8d1df744ce2af90f17d3c7a2621)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:36.790796 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86853bf7-617b-46c9-9741-f479c86f55af admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:36.897898 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-86853bf7-617b-46c9-9741-f479c86f55af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c901e8d1df744ce2af90f17d3c7a2621'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:36.900024 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-86853bf7-617b-46c9-9741-f479c86f55af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c901e8d1df744ce2af90f17d3c7a2621'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:36.945771 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-86853bf7-617b-46c9-9741-f479c86f55af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c901e8d1df744ce2af90f17d3c7a2621'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:36.970856 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-86853bf7-617b-46c9-9741-f479c86f55af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c901e8d1df744ce2af90f17d3c7a2621'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:36.993805 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-86853bf7-617b-46c9-9741-f479c86f55af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c901e8d1df744ce2af90f17d3c7a2621'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:37.014667 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 865/1725] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:15:36 2022] DELETE /identity/v3/users/c901e8d1df744ce2af90f17d3c7a2621 => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:37.167643 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af863d-bc81-49ed-a0e5-809fca07bcdf None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:37.168474 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af863d-bc81-49ed-a0e5-809fca07bcdf None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:37.168916 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af863d-bc81-49ed-a0e5-809fca07bcdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:37.210657 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:37.247208 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-d6af863d-bc81-49ed-a0e5-809fca07bcdf None None] MFA Rules not processed for user `5e052654361f4580ab36a347610f63c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:37.249223 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-d6af863d-bc81-49ed-a0e5-809fca07bcdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:37.332366 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e92a61-19e5-45dd-9963-ffea1c8df797 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:37.341043 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e92a61-19e5-45dd-9963-ffea1c8df797 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:37.341410 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e92a61-19e5-45dd-9963-ffea1c8df797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:37.394255 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 861/1726] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:37.448849 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-84e92a61-19e5-45dd-9963-ffea1c8df797 None None] MFA Rules not processed for user `a32702a67a7b49ffa153fc59251fd169`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:37.450127 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-84e92a61-19e5-45dd-9963-ffea1c8df797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:37.455313 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:37.494633 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af863d-bc81-49ed-a0e5-809fca07bcdf None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:37.496188 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-d6af863d-bc81-49ed-a0e5-809fca07bcdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:37.558310 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 866/1727] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Jul 13 23:15:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:37.837550 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e92a61-19e5-45dd-9963-ffea1c8df797 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:37.839815 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-84e92a61-19e5-45dd-9963-ffea1c8df797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:37.874128 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af863d-bc81-49ed-a0e5-809fca07bcdf None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:37.904130 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6159d0a-1009-41c5-a95c-5f781a3b4cea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:37.905464 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6159d0a-1009-41c5-a95c-5f781a3b4cea service nova] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:37.905778 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6159d0a-1009-41c5-a95c-5f781a3b4cea service nova] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:37.906088 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6159d0a-1009-41c5-a95c-5f781a3b4cea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:37.916141 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-d6159d0a-1009-41c5-a95c-5f781a3b4cea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:38.211872 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6159d0a-1009-41c5-a95c-5f781a3b4cea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:38.220264 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6159d0a-1009-41c5-a95c-5f781a3b4cea service nova] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:38.236835 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 862/1728] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:15:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 743 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:15:38.247702 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e92a61-19e5-45dd-9963-ffea1c8df797 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:38.274489 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed9af91-6afe-457b-9fa9-24be49a93a78 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786fc44611804d3ca6280ffcab415569', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:38.275339 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed9af91-6afe-457b-9fa9-24be49a93a78 service swift] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:38.275555 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed9af91-6afe-457b-9fa9-24be49a93a78 service swift] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:38.276057 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed9af91-6afe-457b-9fa9-24be49a93a78 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:38.299720 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed9af91-6afe-457b-9fa9-24be49a93a78 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:38.304601 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed9af91-6afe-457b-9fa9-24be49a93a78 service swift] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:38.366015 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6159d0a-1009-41c5-a95c-5f781a3b4cea service nova] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:38.366319 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 867/1729] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Jul 13 23:15:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 23:15:38.368108 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-d6159d0a-1009-41c5-a95c-5f781a3b4cea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:38.681979 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:38.779197 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6159d0a-1009-41c5-a95c-5f781a3b4cea service nova] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:38.800288 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e977c69-0f5e-4e7e-9ca9-2858deec43a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:38.801246 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e977c69-0f5e-4e7e-9ca9-2858deec43a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:38.801527 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e977c69-0f5e-4e7e-9ca9-2858deec43a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:38.801856 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e977c69-0f5e-4e7e-9ca9-2858deec43a8 admin admin] PATH_INFO: `/v3/users/338087db4ab9474ca0147aaa200e16fa` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:38.813715 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e977c69-0f5e-4e7e-9ca9-2858deec43a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=338087db4ab9474ca0147aaa200e16fa)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:38.824004 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e977c69-0f5e-4e7e-9ca9-2858deec43a8 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:38.960380 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-4e977c69-0f5e-4e7e-9ca9-2858deec43a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '338087db4ab9474ca0147aaa200e16fa'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:38.962093 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-4e977c69-0f5e-4e7e-9ca9-2858deec43a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '338087db4ab9474ca0147aaa200e16fa'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:39.026120 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-4e977c69-0f5e-4e7e-9ca9-2858deec43a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '338087db4ab9474ca0147aaa200e16fa'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:39.066439 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-4e977c69-0f5e-4e7e-9ca9-2858deec43a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '338087db4ab9474ca0147aaa200e16fa'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:39.069794 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-4e977c69-0f5e-4e7e-9ca9-2858deec43a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '338087db4ab9474ca0147aaa200e16fa'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:39.115885 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 863/1730] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:15:38 2022] DELETE /identity/v3/users/338087db4ab9474ca0147aaa200e16fa => generated 0 bytes in 750 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:39.120936 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd1998-6d71-4c01-a882-10e4c06636bf None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:39.122673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd1998-6d71-4c01-a882-10e4c06636bf None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:39.124252 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd1998-6d71-4c01-a882-10e4c06636bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:39.197641 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-aebd1998-6d71-4c01-a882-10e4c06636bf None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:39.198991 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-aebd1998-6d71-4c01-a882-10e4c06636bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:39.512092 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12957ac8-81d3-4bac-b031-7355023c027a None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:39.512736 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12957ac8-81d3-4bac-b031-7355023c027a None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:39.512736 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12957ac8-81d3-4bac-b031-7355023c027a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:39.588776 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-12957ac8-81d3-4bac-b031-7355023c027a None None] MFA Rules not processed for user `e2e2bd6cf0a44d529ef9b2678c467aba`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:39.597552 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-12957ac8-81d3-4bac-b031-7355023c027a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:39.667305 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 868/1731] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 547 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:39.868743 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd1998-6d71-4c01-a882-10e4c06636bf None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:39.870239 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-aebd1998-6d71-4c01-a882-10e4c06636bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:39.913463 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 864/1732] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 402 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:39.938924 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:40.128846 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd1998-6d71-4c01-a882-10e4c06636bf None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:40.159808 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d52a9c-71c3-4027-8cff-b0718832f412 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:40.160499 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d52a9c-71c3-4027-8cff-b0718832f412 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:40.164274 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d52a9c-71c3-4027-8cff-b0718832f412 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:40.164573 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d52a9c-71c3-4027-8cff-b0718832f412 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:40.218872 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d52a9c-71c3-4027-8cff-b0718832f412 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:40.228035 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d52a9c-71c3-4027-8cff-b0718832f412 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:40.274851 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 869/1733] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:15:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:15:40.974032 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12957ac8-81d3-4bac-b031-7355023c027a None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:40.976826 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-12957ac8-81d3-4bac-b031-7355023c027a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:41.038707 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12957ac8-81d3-4bac-b031-7355023c027a None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:41.079314 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e47879-c278-4fe7-ba06-7b9fb2ff2ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:41.080241 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e47879-c278-4fe7-ba06-7b9fb2ff2ec1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:41.080524 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e47879-c278-4fe7-ba06-7b9fb2ff2ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:41.080800 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e47879-c278-4fe7-ba06-7b9fb2ff2ec1 admin admin] PATH_INFO: `/v3/projects/a4eb70d5598a4ff6962a8f0170a8b3b8` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:41.085092 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e47879-c278-4fe7-ba06-7b9fb2ff2ec1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4eb70d5598a4ff6962a8f0170a8b3b8)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:41.098769 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e47879-c278-4fe7-ba06-7b9fb2ff2ec1 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:41.276284 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-c0e47879-c278-4fe7-ba06-7b9fb2ff2ec1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4eb70d5598a4ff6962a8f0170a8b3b8', 'request_id': 'req-c0e47879-c278-4fe7-ba06-7b9fb2ff2ec1'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:41.289215 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0f270-4e41-4d6d-984c-3f1010845796 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:41.289723 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0f270-4e41-4d6d-984c-3f1010845796 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:41.289817 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0f270-4e41-4d6d-984c-3f1010845796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:41.316557 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-c0e47879-c278-4fe7-ba06-7b9fb2ff2ec1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4eb70d5598a4ff6962a8f0170a8b3b8', 'request_id': 'req-c0e47879-c278-4fe7-ba06-7b9fb2ff2ec1'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:41.341563 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-c0e47879-c278-4fe7-ba06-7b9fb2ff2ec1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4eb70d5598a4ff6962a8f0170a8b3b8', 'request_id': 'req-c0e47879-c278-4fe7-ba06-7b9fb2ff2ec1'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:41.345372 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-ffc0f270-4e41-4d6d-984c-3f1010845796 None None] MFA Rules not processed for user `2d670df2aff5478ebf61f55358d554de`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:41.349421 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ffc0f270-4e41-4d6d-984c-3f1010845796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:41.369652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:41.391692 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-c0e47879-c278-4fe7-ba06-7b9fb2ff2ec1 admin admin] The token cache is being invalidate because project a4eb70d5598a4ff6962a8f0170a8b3b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:15:41.392003 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-c0e47879-c278-4fe7-ba06-7b9fb2ff2ec1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:41.394862 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-c0e47879-c278-4fe7-ba06-7b9fb2ff2ec1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:41.402800 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 865/1734] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:15:40 2022] DELETE /identity/v3/projects/a4eb70d5598a4ff6962a8f0170a8b3b8 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:41.404691 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cebd2cf-9474-42b3-8112-0c4551ac0bae None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:41.404985 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cebd2cf-9474-42b3-8112-0c4551ac0bae None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:41.405261 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cebd2cf-9474-42b3-8112-0c4551ac0bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:41.483262 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-0cebd2cf-9474-42b3-8112-0c4551ac0bae None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:41.492735 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-0cebd2cf-9474-42b3-8112-0c4551ac0bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:41.705558 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 870/1735] 10.222.0.58 () {58 vars in 1015 bytes} [Wed Jul 13 23:15:41 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 418 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:15:41.739114 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da4a4ce-fb09-4506-abfe-672a5c3c08ab None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:41.739598 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da4a4ce-fb09-4506-abfe-672a5c3c08ab None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:41.740128 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da4a4ce-fb09-4506-abfe-672a5c3c08ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:41.742004 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 866/1736] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:15:41.803198 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-9da4a4ce-fb09-4506-abfe-672a5c3c08ab None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:41.807266 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9da4a4ce-fb09-4506-abfe-672a5c3c08ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:41.843342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 871/1737] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:41.847883 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cebd2cf-9474-42b3-8112-0c4551ac0bae None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:41.858820 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cebd2cf-9474-42b3-8112-0c4551ac0bae None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:41.883241 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5404361-7a32-4de6-836e-0a9dd2a0c86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:41.884242 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5404361-7a32-4de6-836e-0a9dd2a0c86e admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:41.884681 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5404361-7a32-4de6-836e-0a9dd2a0c86e admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:41.885102 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5404361-7a32-4de6-836e-0a9dd2a0c86e admin admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:41.888188 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5404361-7a32-4de6-836e-0a9dd2a0c86e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:15:41.889272 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5404361-7a32-4de6-836e-0a9dd2a0c86e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:41.896409 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5404361-7a32-4de6-836e-0a9dd2a0c86e admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:41.962248 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 867/1738] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:15:41.963577 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da4a4ce-fb09-4506-abfe-672a5c3c08ab None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:41.970180 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5404361-7a32-4de6-836e-0a9dd2a0c86e admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:41.977894 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da4a4ce-fb09-4506-abfe-672a5c3c08ab None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:41.984179 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-e5404361-7a32-4de6-836e-0a9dd2a0c86e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:42.029184 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655de47c-c3a0-4008-bb5d-2440d7df12e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:42.030848 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655de47c-c3a0-4008-bb5d-2440d7df12e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:42.036125 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655de47c-c3a0-4008-bb5d-2440d7df12e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:42.038026 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655de47c-c3a0-4008-bb5d-2440d7df12e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:42.040244 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655de47c-c3a0-4008-bb5d-2440d7df12e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:42.055342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655de47c-c3a0-4008-bb5d-2440d7df12e9 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:42.081248 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5404361-7a32-4de6-836e-0a9dd2a0c86e admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:42.099103 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd76810e-279a-4d6d-8b45-fd324def669f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:42.099833 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76810e-279a-4d6d-8b45-fd324def669f service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:42.100145 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76810e-279a-4d6d-8b45-fd324def669f service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:42.100566 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76810e-279a-4d6d-8b45-fd324def669f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:42.120512 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd76810e-279a-4d6d-8b45-fd324def669f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:42.128830 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd76810e-279a-4d6d-8b45-fd324def669f service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:42.159176 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 872/1739] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:15:41 2022] POST /identity/v3/projects => generated 377 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:42.160189 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655de47c-c3a0-4008-bb5d-2440d7df12e9 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:42.169573 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 868/1740] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:15:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:15:42.322868 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655de47c-c3a0-4008-bb5d-2440d7df12e9 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:42.354603 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7574cd-5fae-4cdd-88f7-26c66c777962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:42.356933 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7574cd-5fae-4cdd-88f7-26c66c777962 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:42.358779 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7574cd-5fae-4cdd-88f7-26c66c777962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:42.366550 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7574cd-5fae-4cdd-88f7-26c66c777962 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:42.367579 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7574cd-5fae-4cdd-88f7-26c66c777962 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:42.374036 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7574cd-5fae-4cdd-88f7-26c66c777962 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:42.456313 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd76810e-279a-4d6d-8b45-fd324def669f service neutron] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:42.461626 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 873/1741] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:15:42 2022] POST /identity/v3/users => generated 289 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:42.491345 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd76810e-279a-4d6d-8b45-fd324def669f service neutron] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:42.498783 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0f7630-eeff-4e79-bb7c-e07b76bc3cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:42.499935 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f7630-eeff-4e79-bb7c-e07b76bc3cdb admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:42.500450 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f7630-eeff-4e79-bb7c-e07b76bc3cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:42.500959 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f7630-eeff-4e79-bb7c-e07b76bc3cdb admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:42.501926 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0f7630-eeff-4e79-bb7c-e07b76bc3cdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:42.504017 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0f7630-eeff-4e79-bb7c-e07b76bc3cdb admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:42.538585 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7574cd-5fae-4cdd-88f7-26c66c777962 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:42.538916 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 869/1742] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:42.556807 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7574cd-5fae-4cdd-88f7-26c66c777962 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:42.585199 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29e2d2-9cdd-430d-8524-95382cf46b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:42.586220 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29e2d2-9cdd-430d-8524-95382cf46b7e admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:42.586682 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29e2d2-9cdd-430d-8524-95382cf46b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:42.587137 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29e2d2-9cdd-430d-8524-95382cf46b7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:42.595791 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b29e2d2-9cdd-430d-8524-95382cf46b7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:42.603586 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b29e2d2-9cdd-430d-8524-95382cf46b7e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:42.668760 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 874/1743] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:42.669214 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0f7630-eeff-4e79-bb7c-e07b76bc3cdb admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:42.690240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0f7630-eeff-4e79-bb7c-e07b76bc3cdb admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:42.699260 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863dce46-0572-4a94-972a-b502695e10dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:42.701995 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863dce46-0572-4a94-972a-b502695e10dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:42.702515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863dce46-0572-4a94-972a-b502695e10dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:42.703952 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863dce46-0572-4a94-972a-b502695e10dd admin admin] PATH_INFO: `/v3/projects/729e4e5bcaae4189b6eabc30643ed12a/users/1e12088c833f4df885e0e2ab4de2d757/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:42.757924 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863dce46-0572-4a94-972a-b502695e10dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=729e4e5bcaae4189b6eabc30643ed12a, user_id=1e12088c833f4df885e0e2ab4de2d757, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:42.766290 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863dce46-0572-4a94-972a-b502695e10dd admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:42.801497 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 870/1744] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:15:42 2022] PUT /identity/v3/projects/729e4e5bcaae4189b6eabc30643ed12a/users/1e12088c833f4df885e0e2ab4de2d757/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:42.814714 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccff41-bd8a-4009-81a7-de54c3789295 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:42.815008 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccff41-bd8a-4009-81a7-de54c3789295 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:42.815288 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccff41-bd8a-4009-81a7-de54c3789295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:42.894502 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-e7ccff41-bd8a-4009-81a7-de54c3789295 None None] MFA Rules not processed for user `1e12088c833f4df885e0e2ab4de2d757`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:42.901679 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e7ccff41-bd8a-4009-81a7-de54c3789295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:42.919763 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:43.225317 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863dce46-0572-4a94-972a-b502695e10dd admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:43.227313 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 875/1745] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 413 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:43.526220 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863dce46-0572-4a94-972a-b502695e10dd admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:43.546223 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e2823a-7af7-409b-b6ab-85db51163deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:43.547115 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2823a-7af7-409b-b6ab-85db51163deb admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:43.547402 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2823a-7af7-409b-b6ab-85db51163deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:43.548229 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2823a-7af7-409b-b6ab-85db51163deb admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:43.548535 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e2823a-7af7-409b-b6ab-85db51163deb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:43.553199 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e2823a-7af7-409b-b6ab-85db51163deb admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:43.656577 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 871/1746] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:15:43 2022] POST /identity/v3/projects => generated 375 bytes in 432 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:43.660699 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ccff41-bd8a-4009-81a7-de54c3789295 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:43.684726 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ccff41-bd8a-4009-81a7-de54c3789295 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:43.901992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e2823a-7af7-409b-b6ab-85db51163deb admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:43.904941 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-24e2823a-7af7-409b-b6ab-85db51163deb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:43.946921 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7b209-e3d3-42e0-a853-2c7ea0672d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:43.947654 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7b209-e3d3-42e0-a853-2c7ea0672d62 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:43.951656 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7b209-e3d3-42e0-a853-2c7ea0672d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:43.952012 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7b209-e3d3-42e0-a853-2c7ea0672d62 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:43.953841 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a7b209-e3d3-42e0-a853-2c7ea0672d62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:43.958520 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a7b209-e3d3-42e0-a853-2c7ea0672d62 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:44.018647 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:44.042746 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 876/1747] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:15:43 2022] POST /identity/v3/users => generated 294 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:44.046163 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7b209-e3d3-42e0-a853-2c7ea0672d62 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:44.078925 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7b209-e3d3-42e0-a853-2c7ea0672d62 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:44.109096 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef166c0-a014-40d2-b654-f971eed9d2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:44.109785 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef166c0-a014-40d2-b654-f971eed9d2c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:44.110074 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef166c0-a014-40d2-b654-f971eed9d2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:44.117055 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef166c0-a014-40d2-b654-f971eed9d2c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:44.119305 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef166c0-a014-40d2-b654-f971eed9d2c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:44.125069 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef166c0-a014-40d2-b654-f971eed9d2c5 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:44.181833 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 877/1748] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:44 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:44.188626 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef166c0-a014-40d2-b654-f971eed9d2c5 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:44.211231 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e2823a-7af7-409b-b6ab-85db51163deb admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:44.230226 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef166c0-a014-40d2-b654-f971eed9d2c5 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:44.233690 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b534791-b373-435a-a405-83231c01526f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:44.238500 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b534791-b373-435a-a405-83231c01526f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:44.238927 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b534791-b373-435a-a405-83231c01526f admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:44.240450 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b534791-b373-435a-a405-83231c01526f admin admin] PATH_INFO: `/v3/projects/42e47b1e95f64d77ac187a819d8be298` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:44.248214 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b534791-b373-435a-a405-83231c01526f admin admin] RBAC: Authorizing `identity:delete_project(project_id=42e47b1e95f64d77ac187a819d8be298)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:44.248689 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd4f5aa-1e28-40db-a67f-423524e215da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:44.249675 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4f5aa-1e28-40db-a67f-423524e215da admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:44.258858 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b534791-b373-435a-a405-83231c01526f admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:44.271186 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4f5aa-1e28-40db-a67f-423524e215da admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:44.271513 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4f5aa-1e28-40db-a67f-423524e215da admin admin] PATH_INFO: `/v3/projects/a61a6cf8b59a42a2880d49cb0ccc3432/users/04a550e3830e4655a724f952c2199103/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:44.343623 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd4f5aa-1e28-40db-a67f-423524e215da admin admin] RBAC: Authorizing `identity:create_grant(project_id=a61a6cf8b59a42a2880d49cb0ccc3432, user_id=04a550e3830e4655a724f952c2199103, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:44.356832 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd4f5aa-1e28-40db-a67f-423524e215da admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:44.402174 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 878/1749] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:15:44 2022] PUT /identity/v3/projects/a61a6cf8b59a42a2880d49cb0ccc3432/users/04a550e3830e4655a724f952c2199103/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:44.403191 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd4f5aa-1e28-40db-a67f-423524e215da admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:44.413102 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-2b534791-b373-435a-a405-83231c01526f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42e47b1e95f64d77ac187a819d8be298', 'request_id': 'req-2b534791-b373-435a-a405-83231c01526f'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:44.428845 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-2b534791-b373-435a-a405-83231c01526f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42e47b1e95f64d77ac187a819d8be298', 'request_id': 'req-2b534791-b373-435a-a405-83231c01526f'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:44.448828 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-2b534791-b373-435a-a405-83231c01526f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42e47b1e95f64d77ac187a819d8be298', 'request_id': 'req-2b534791-b373-435a-a405-83231c01526f'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:44.493972 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-2b534791-b373-435a-a405-83231c01526f admin admin] The token cache is being invalidate because project 42e47b1e95f64d77ac187a819d8be298 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:15:44.495154 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-2b534791-b373-435a-a405-83231c01526f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:44.501234 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-2b534791-b373-435a-a405-83231c01526f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:44.507513 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 872/1750] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:15:43 2022] DELETE /identity/v3/projects/42e47b1e95f64d77ac187a819d8be298 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:15:44.512916 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b534791-b373-435a-a405-83231c01526f admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:44.515010 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-2b534791-b373-435a-a405-83231c01526f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:44.591062 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd4f5aa-1e28-40db-a67f-423524e215da admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:44.618428 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f412525c-0db9-4863-979a-9618c7260a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:44.621172 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412525c-0db9-4863-979a-9618c7260a26 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:44.622011 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412525c-0db9-4863-979a-9618c7260a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:44.622276 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412525c-0db9-4863-979a-9618c7260a26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:44.625489 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f412525c-0db9-4863-979a-9618c7260a26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:44.630373 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f412525c-0db9-4863-979a-9618c7260a26 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:44.639132 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:44.669300 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 879/1751] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:44 2022] GET /identity/v3/roles => generated 1420 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:44.673580 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f412525c-0db9-4863-979a-9618c7260a26 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:44.681754 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f412525c-0db9-4863-979a-9618c7260a26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:44.707833 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f412525c-0db9-4863-979a-9618c7260a26 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:44.724931 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11111f2-0163-4d81-888b-35a3e6bf1a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:44.725862 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11111f2-0163-4d81-888b-35a3e6bf1a1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:44.727055 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11111f2-0163-4d81-888b-35a3e6bf1a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:44.727196 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11111f2-0163-4d81-888b-35a3e6bf1a1b admin admin] PATH_INFO: `/v3/projects/a61a6cf8b59a42a2880d49cb0ccc3432/users/04a550e3830e4655a724f952c2199103/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:44.730266 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11111f2-0163-4d81-888b-35a3e6bf1a1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a61a6cf8b59a42a2880d49cb0ccc3432, user_id=04a550e3830e4655a724f952c2199103, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:44.734509 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11111f2-0163-4d81-888b-35a3e6bf1a1b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:44.797542 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 880/1752] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:15:44 2022] PUT /identity/v3/projects/a61a6cf8b59a42a2880d49cb0ccc3432/users/04a550e3830e4655a724f952c2199103/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:15:44.799432 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789db8d-3c38-46a9-aa03-70400b6f20fe None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:44.799432 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789db8d-3c38-46a9-aa03-70400b6f20fe None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:44.806688 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b534791-b373-435a-a405-83231c01526f admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:44.808155 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789db8d-3c38-46a9-aa03-70400b6f20fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:44.869319 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-f789db8d-3c38-46a9-aa03-70400b6f20fe None None] MFA Rules not processed for user `04a550e3830e4655a724f952c2199103`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:44.871249 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f789db8d-3c38-46a9-aa03-70400b6f20fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:45.000305 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:45.038455 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c8c8c4-ac2d-4c0c-a6e7-5385322687d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:45.039696 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8c8c4-ac2d-4c0c-a6e7-5385322687d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:45.039939 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8c8c4-ac2d-4c0c-a6e7-5385322687d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:45.040318 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8c8c4-ac2d-4c0c-a6e7-5385322687d5 admin admin] PATH_INFO: `/v3/users/44c92e64f7ce4d42ab56e90f06e024fa` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:45.042654 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c8c8c4-ac2d-4c0c-a6e7-5385322687d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44c92e64f7ce4d42ab56e90f06e024fa)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:45.048180 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c8c8c4-ac2d-4c0c-a6e7-5385322687d5 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:45.081896 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 881/1753] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:44 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:45.097444 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f789db8d-3c38-46a9-aa03-70400b6f20fe None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:45.099155 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f789db8d-3c38-46a9-aa03-70400b6f20fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:45.149719 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-99c8c8c4-ac2d-4c0c-a6e7-5385322687d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44c92e64f7ce4d42ab56e90f06e024fa'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:45.152511 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-99c8c8c4-ac2d-4c0c-a6e7-5385322687d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44c92e64f7ce4d42ab56e90f06e024fa'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:45.177573 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-99c8c8c4-ac2d-4c0c-a6e7-5385322687d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44c92e64f7ce4d42ab56e90f06e024fa'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:45.198345 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-99c8c8c4-ac2d-4c0c-a6e7-5385322687d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44c92e64f7ce4d42ab56e90f06e024fa'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:45.201646 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-99c8c8c4-ac2d-4c0c-a6e7-5385322687d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44c92e64f7ce4d42ab56e90f06e024fa'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:45.226944 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 873/1754] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:15:44 2022] DELETE /identity/v3/users/44c92e64f7ce4d42ab56e90f06e024fa => generated 0 bytes in 715 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:45.233240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f789db8d-3c38-46a9-aa03-70400b6f20fe None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:45.292036 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0696ad3a-e3a1-478c-af89-4b261cccf97b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:45.292869 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696ad3a-e3a1-478c-af89-4b261cccf97b service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:45.293263 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696ad3a-e3a1-478c-af89-4b261cccf97b service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:45.293667 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696ad3a-e3a1-478c-af89-4b261cccf97b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:45.295905 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0696ad3a-e3a1-478c-af89-4b261cccf97b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:45.344334 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:45.372388 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0696ad3a-e3a1-478c-af89-4b261cccf97b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:45.377051 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0696ad3a-e3a1-478c-af89-4b261cccf97b service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:45.406561 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 882/1755] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:15:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:15:47.087120 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c8c8c4-ac2d-4c0c-a6e7-5385322687d5 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:47.088684 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-99c8c8c4-ac2d-4c0c-a6e7-5385322687d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:47.276533 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c8c8c4-ac2d-4c0c-a6e7-5385322687d5 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:47.283901 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e6831-3117-4798-8e7d-085b1b4c23d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:47.284933 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e6831-3117-4798-8e7d-085b1b4c23d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:47.285224 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e6831-3117-4798-8e7d-085b1b4c23d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:47.285557 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e6831-3117-4798-8e7d-085b1b4c23d5 admin admin] PATH_INFO: `/v3/projects/fa424e8207354f4b88788563754ca3b6` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:47.286734 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e6831-3117-4798-8e7d-085b1b4c23d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa424e8207354f4b88788563754ca3b6)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:47.288955 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6e6831-3117-4798-8e7d-085b1b4c23d5 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:47.432228 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-4d6e6831-3117-4798-8e7d-085b1b4c23d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa424e8207354f4b88788563754ca3b6', 'request_id': 'req-4d6e6831-3117-4798-8e7d-085b1b4c23d5'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:47.453369 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-4d6e6831-3117-4798-8e7d-085b1b4c23d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa424e8207354f4b88788563754ca3b6', 'request_id': 'req-4d6e6831-3117-4798-8e7d-085b1b4c23d5'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:47.471583 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-4d6e6831-3117-4798-8e7d-085b1b4c23d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa424e8207354f4b88788563754ca3b6', 'request_id': 'req-4d6e6831-3117-4798-8e7d-085b1b4c23d5'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:47.531351 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-4d6e6831-3117-4798-8e7d-085b1b4c23d5 admin admin] The token cache is being invalidate because project fa424e8207354f4b88788563754ca3b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:15:47.531697 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-4d6e6831-3117-4798-8e7d-085b1b4c23d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:47.533642 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-4d6e6831-3117-4798-8e7d-085b1b4c23d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:47.542540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 874/1756] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:15:47 2022] DELETE /identity/v3/projects/fa424e8207354f4b88788563754ca3b6 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:47.546884 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633625c-46ab-40f8-809c-7053e43e3c86 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:47.547807 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633625c-46ab-40f8-809c-7053e43e3c86 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:47.548099 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633625c-46ab-40f8-809c-7053e43e3c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:47.577858 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-a633625c-46ab-40f8-809c-7053e43e3c86 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:47.579461 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a633625c-46ab-40f8-809c-7053e43e3c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:47.741743 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 883/1757] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:47.745857 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96141e6f-df10-458c-a51a-0142cf6b7419 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:47.746395 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96141e6f-df10-458c-a51a-0142cf6b7419 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:47.750647 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96141e6f-df10-458c-a51a-0142cf6b7419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:47.813640 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:47.851162 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-96141e6f-df10-458c-a51a-0142cf6b7419 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:47.852479 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-96141e6f-df10-458c-a51a-0142cf6b7419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:47.904530 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 875/1758] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:47.908804 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a633625c-46ab-40f8-809c-7053e43e3c86 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:47.920836 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a633625c-46ab-40f8-809c-7053e43e3c86 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:47.940418 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e2eda-dc95-4362-948d-558d89789a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:47.941031 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e2eda-dc95-4362-948d-558d89789a8f admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:47.941416 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e2eda-dc95-4362-948d-558d89789a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:47.941698 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e2eda-dc95-4362-948d-558d89789a8f admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:47.942597 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542e2eda-dc95-4362-948d-558d89789a8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:15:47.942852 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542e2eda-dc95-4362-948d-558d89789a8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:47.944711 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542e2eda-dc95-4362-948d-558d89789a8f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:47.986093 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 884/1759] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:15:47.993016 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96141e6f-df10-458c-a51a-0142cf6b7419 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:48.008222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96141e6f-df10-458c-a51a-0142cf6b7419 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:48.027939 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f8bc79-3b87-47aa-90bc-7276891c0a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:48.028495 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f8bc79-3b87-47aa-90bc-7276891c0a03 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:48.029306 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f8bc79-3b87-47aa-90bc-7276891c0a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:48.029648 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f8bc79-3b87-47aa-90bc-7276891c0a03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:48.030271 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f8bc79-3b87-47aa-90bc-7276891c0a03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:48.033205 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f8bc79-3b87-47aa-90bc-7276891c0a03 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:48.104399 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 876/1760] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:15:47 2022] POST /identity/v3/projects => generated 373 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:48.109154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e2eda-dc95-4362-948d-558d89789a8f admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:48.124904 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e2eda-dc95-4362-948d-558d89789a8f admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:48.302244 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27704f62-77f7-499c-aac8-1e264271304b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:48.303099 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27704f62-77f7-499c-aac8-1e264271304b admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:48.303389 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27704f62-77f7-499c-aac8-1e264271304b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:48.303896 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27704f62-77f7-499c-aac8-1e264271304b admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:48.307918 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27704f62-77f7-499c-aac8-1e264271304b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:48.321045 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27704f62-77f7-499c-aac8-1e264271304b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:48.332518 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191b99a-aa4e-44bc-8332-2d591357dd08 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:48.332835 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191b99a-aa4e-44bc-8332-2d591357dd08 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:48.333190 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191b99a-aa4e-44bc-8332-2d591357dd08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:48.360362 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-1191b99a-aa4e-44bc-8332-2d591357dd08 None None] MFA Rules not processed for user `75652cd3bfa04c7ebae2cbea77cb5fd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:48.362399 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-1191b99a-aa4e-44bc-8332-2d591357dd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:48.464209 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 885/1761] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:15:48 2022] POST /identity/v3/users => generated 287 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:48.471815 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27704f62-77f7-499c-aac8-1e264271304b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:48.483585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 877/1762] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:48.514267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27704f62-77f7-499c-aac8-1e264271304b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:48.539296 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3abeec-b18d-4602-9831-fc3a2ceab3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:48.540206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3abeec-b18d-4602-9831-fc3a2ceab3e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:48.540698 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3abeec-b18d-4602-9831-fc3a2ceab3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:48.541082 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3abeec-b18d-4602-9831-fc3a2ceab3e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:48.542060 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3abeec-b18d-4602-9831-fc3a2ceab3e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:48.554245 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3abeec-b18d-4602-9831-fc3a2ceab3e6 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:48.597598 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1191b99a-aa4e-44bc-8332-2d591357dd08 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:48.598611 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 886/1763] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:15:48.624770 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1191b99a-aa4e-44bc-8332-2d591357dd08 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:48.642549 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c685413-a099-48ef-a718-3e9c57764a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:48.643299 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c685413-a099-48ef-a718-3e9c57764a3b admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:48.643620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c685413-a099-48ef-a718-3e9c57764a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:48.643906 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c685413-a099-48ef-a718-3e9c57764a3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:48.645325 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c685413-a099-48ef-a718-3e9c57764a3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:48.652642 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c685413-a099-48ef-a718-3e9c57764a3b admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:48.683007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 878/1764] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:48.687021 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3abeec-b18d-4602-9831-fc3a2ceab3e6 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:48.700128 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3abeec-b18d-4602-9831-fc3a2ceab3e6 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:48.726509 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580490ec-921d-4d90-b807-d5e6e03d2f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:48.727826 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580490ec-921d-4d90-b807-d5e6e03d2f12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:48.728346 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580490ec-921d-4d90-b807-d5e6e03d2f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:48.728854 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580490ec-921d-4d90-b807-d5e6e03d2f12 admin admin] PATH_INFO: `/v3/projects/675293a8cf7e48bc9adcb602b2a17d9f/users/30cbc5b51aad494199815bcccb909b1e/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:48.784129 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580490ec-921d-4d90-b807-d5e6e03d2f12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=675293a8cf7e48bc9adcb602b2a17d9f, user_id=30cbc5b51aad494199815bcccb909b1e, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:48.792316 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580490ec-921d-4d90-b807-d5e6e03d2f12 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:48.837945 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 887/1765] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:15:48 2022] PUT /identity/v3/projects/675293a8cf7e48bc9adcb602b2a17d9f/users/30cbc5b51aad494199815bcccb909b1e/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:48.850172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8211e673-f8df-4140-bb82-584bff5156a1 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:48.850464 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8211e673-f8df-4140-bb82-584bff5156a1 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:48.850751 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8211e673-f8df-4140-bb82-584bff5156a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:48.932121 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-8211e673-f8df-4140-bb82-584bff5156a1 None None] MFA Rules not processed for user `30cbc5b51aad494199815bcccb909b1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:48.933767 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-8211e673-f8df-4140-bb82-584bff5156a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:49.118358 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 879/1766] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:49.119913 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580490ec-921d-4d90-b807-d5e6e03d2f12 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:49.160433 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:49.249263 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd202df-8485-4289-9389-e6a66d410adc None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:49.249263 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd202df-8485-4289-9389-e6a66d410adc None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:49.249515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd202df-8485-4289-9389-e6a66d410adc None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:49.251602 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 880/1767] 10.222.0.60 () {54 vars in 921 bytes} [Wed Jul 13 23:15:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 23:15:49.257257 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580490ec-921d-4d90-b807-d5e6e03d2f12 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:49.277720 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a10308-cfed-4ffc-9f41-bb89e974c84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:49.280982 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a10308-cfed-4ffc-9f41-bb89e974c84e admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:49.281451 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a10308-cfed-4ffc-9f41-bb89e974c84e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:49.281810 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a10308-cfed-4ffc-9f41-bb89e974c84e admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:49.282647 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a10308-cfed-4ffc-9f41-bb89e974c84e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:49.285920 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a10308-cfed-4ffc-9f41-bb89e974c84e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:49.291962 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b7136-4491-418e-9912-cf6826145a44 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:49.292395 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b7136-4491-418e-9912-cf6826145a44 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:49.292785 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b7136-4491-418e-9912-cf6826145a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:49.334006 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-c37b7136-4491-418e-9912-cf6826145a44 None None] MFA Rules not processed for user `4789384d158b4995bd30e1e37e30f5f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:49.335468 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-c37b7136-4491-418e-9912-cf6826145a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:49.357447 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 888/1768] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:15:49 2022] POST /identity/v3/projects => generated 371 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:49.368914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a10308-cfed-4ffc-9f41-bb89e974c84e admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:49.479688 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:49.515074 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 881/1769] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Jul 13 23:15:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:49.521900 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a10308-cfed-4ffc-9f41-bb89e974c84e admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:49.536959 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45833627-9073-41eb-9dc8-7abec3bd86d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:49.538270 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45833627-9073-41eb-9dc8-7abec3bd86d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:49.538270 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45833627-9073-41eb-9dc8-7abec3bd86d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:49.559218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45833627-9073-41eb-9dc8-7abec3bd86d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:49.560096 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45833627-9073-41eb-9dc8-7abec3bd86d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:49.562664 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45833627-9073-41eb-9dc8-7abec3bd86d7 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:49.592976 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 889/1770] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:15:49 2022] POST /identity/v3/users => generated 292 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:15:49.597170 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37b7136-4491-418e-9912-cf6826145a44 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:49.614852 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37b7136-4491-418e-9912-cf6826145a44 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:49.625073 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e89f37-4473-46bd-b6fb-faee7db58810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:49.625774 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e89f37-4473-46bd-b6fb-faee7db58810 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:49.626059 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e89f37-4473-46bd-b6fb-faee7db58810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:49.626336 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e89f37-4473-46bd-b6fb-faee7db58810 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:49.627096 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e89f37-4473-46bd-b6fb-faee7db58810 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:49.629013 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e89f37-4473-46bd-b6fb-faee7db58810 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:49.654825 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 882/1771] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:49.659169 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e89f37-4473-46bd-b6fb-faee7db58810 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:49.659979 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45833627-9073-41eb-9dc8-7abec3bd86d7 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:49.661939 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-38e89f37-4473-46bd-b6fb-faee7db58810 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:49.684475 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45833627-9073-41eb-9dc8-7abec3bd86d7 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:49.695438 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a216019a-8639-4c6a-90f2-2e54f4d63c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:49.696756 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216019a-8639-4c6a-90f2-2e54f4d63c31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:49.698983 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216019a-8639-4c6a-90f2-2e54f4d63c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:49.700171 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216019a-8639-4c6a-90f2-2e54f4d63c31 admin admin] PATH_INFO: `/v3/projects/e45f472a49e3443eae6952b8037e5f5f/users/9ebfc8fcefdd4d7b96dfc3b0720bd1b5/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:49.731226 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a216019a-8639-4c6a-90f2-2e54f4d63c31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e45f472a49e3443eae6952b8037e5f5f, user_id=9ebfc8fcefdd4d7b96dfc3b0720bd1b5, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:49.742770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a216019a-8639-4c6a-90f2-2e54f4d63c31 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:49.766923 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 890/1772] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:15:49 2022] PUT /identity/v3/projects/e45f472a49e3443eae6952b8037e5f5f/users/9ebfc8fcefdd4d7b96dfc3b0720bd1b5/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:49.770622 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a216019a-8639-4c6a-90f2-2e54f4d63c31 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:49.822477 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e89f37-4473-46bd-b6fb-faee7db58810 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:49.877086 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a216019a-8639-4c6a-90f2-2e54f4d63c31 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:49.882889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b707c205-6a4a-481a-8c3a-52918361882e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:49.884176 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707c205-6a4a-481a-8c3a-52918361882e admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:49.884610 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707c205-6a4a-481a-8c3a-52918361882e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:49.885192 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707c205-6a4a-481a-8c3a-52918361882e admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:49.886838 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b707c205-6a4a-481a-8c3a-52918361882e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:49.889435 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b707c205-6a4a-481a-8c3a-52918361882e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:49.901894 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 891/1773] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:49.906392 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b707c205-6a4a-481a-8c3a-52918361882e admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:49.921144 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b707c205-6a4a-481a-8c3a-52918361882e admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:49.926522 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cff223-6c02-4cf3-94ed-16dda6c90d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:49.927416 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cff223-6c02-4cf3-94ed-16dda6c90d5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:49.927726 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cff223-6c02-4cf3-94ed-16dda6c90d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:49.928003 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cff223-6c02-4cf3-94ed-16dda6c90d5b admin admin] PATH_INFO: `/v3/projects/e45f472a49e3443eae6952b8037e5f5f/users/9ebfc8fcefdd4d7b96dfc3b0720bd1b5/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:49.929340 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b967ad-da1b-464b-84e6-98616ffd064d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:49.930045 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cff223-6c02-4cf3-94ed-16dda6c90d5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e45f472a49e3443eae6952b8037e5f5f, user_id=9ebfc8fcefdd4d7b96dfc3b0720bd1b5, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:49.931329 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b967ad-da1b-464b-84e6-98616ffd064d service nova] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:49.931897 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b967ad-da1b-464b-84e6-98616ffd064d service nova] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:49.932555 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b967ad-da1b-464b-84e6-98616ffd064d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:49.933812 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cff223-6c02-4cf3-94ed-16dda6c90d5b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:49.938260 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:49.948661 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b967ad-da1b-464b-84e6-98616ffd064d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:49.951115 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b967ad-da1b-464b-84e6-98616ffd064d service nova] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:49.955705 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 892/1774] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:15:49 2022] PUT /identity/v3/projects/e45f472a49e3443eae6952b8037e5f5f/users/9ebfc8fcefdd4d7b96dfc3b0720bd1b5/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:49.962477 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923481fe-5cdb-41f4-930b-9a03af4c549c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:49.962771 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923481fe-5cdb-41f4-930b-9a03af4c549c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:49.963056 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923481fe-5cdb-41f4-930b-9a03af4c549c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:49.986416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 883/1775] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:15:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:15:50.009505 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-923481fe-5cdb-41f4-930b-9a03af4c549c None None] MFA Rules not processed for user `9ebfc8fcefdd4d7b96dfc3b0720bd1b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:50.013595 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-923481fe-5cdb-41f4-930b-9a03af4c549c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:50.121833 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 893/1776] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:49 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:15:50.134956 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b967ad-da1b-464b-84e6-98616ffd064d service nova] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:50.136436 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-99b967ad-da1b-464b-84e6-98616ffd064d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:50.225782 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b967ad-da1b-464b-84e6-98616ffd064d service nova] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:50.254642 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:50.294843 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e046e-ac4d-4ce9-a668-46bec732bcb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:50.295514 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e046e-ac4d-4ce9-a668-46bec732bcb1 service cinder] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:50.295897 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e046e-ac4d-4ce9-a668-46bec732bcb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:50.296209 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e046e-ac4d-4ce9-a668-46bec732bcb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:50.317548 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976e046e-ac4d-4ce9-a668-46bec732bcb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:50.321071 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976e046e-ac4d-4ce9-a668-46bec732bcb1 service cinder] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:50.343687 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 884/1777] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:15:50 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:15:50.841063 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923481fe-5cdb-41f4-930b-9a03af4c549c None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:50.922206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923481fe-5cdb-41f4-930b-9a03af4c549c None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:50.932452 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fbb51c-54ce-44ed-9b2b-7744503a3a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:50.933355 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fbb51c-54ce-44ed-9b2b-7744503a3a8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:50.933799 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fbb51c-54ce-44ed-9b2b-7744503a3a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:50.934196 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fbb51c-54ce-44ed-9b2b-7744503a3a8e admin admin] PATH_INFO: `/v3/users/30cbc5b51aad494199815bcccb909b1e` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:50.935464 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fbb51c-54ce-44ed-9b2b-7744503a3a8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=30cbc5b51aad494199815bcccb909b1e)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:50.937419 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fbb51c-54ce-44ed-9b2b-7744503a3a8e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:50.993398 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-30fbb51c-54ce-44ed-9b2b-7744503a3a8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30cbc5b51aad494199815bcccb909b1e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:50.994072 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-30fbb51c-54ce-44ed-9b2b-7744503a3a8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30cbc5b51aad494199815bcccb909b1e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:51.007170 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-30fbb51c-54ce-44ed-9b2b-7744503a3a8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30cbc5b51aad494199815bcccb909b1e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:51.014852 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-30fbb51c-54ce-44ed-9b2b-7744503a3a8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30cbc5b51aad494199815bcccb909b1e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:51.023757 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-30fbb51c-54ce-44ed-9b2b-7744503a3a8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30cbc5b51aad494199815bcccb909b1e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:51.026279 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:51.031991 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 894/1778] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:15:50 2022] DELETE /identity/v3/users/30cbc5b51aad494199815bcccb909b1e => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:51.036643 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675a521-ecfe-4e42-af74-b6ab568f5c85 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:51.036957 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675a521-ecfe-4e42-af74-b6ab568f5c85 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:51.037235 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675a521-ecfe-4e42-af74-b6ab568f5c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:51.057278 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-f675a521-ecfe-4e42-af74-b6ab568f5c85 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:51.058476 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-f675a521-ecfe-4e42-af74-b6ab568f5c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:51.124004 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 885/1779] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:51.133800 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fbb51c-54ce-44ed-9b2b-7744503a3a8e admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:51.135272 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-30fbb51c-54ce-44ed-9b2b-7744503a3a8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:51.209859 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fbb51c-54ce-44ed-9b2b-7744503a3a8e admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:51.213775 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35066d2e-13cb-459f-81d6-a837ca27723f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:51.214420 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35066d2e-13cb-459f-81d6-a837ca27723f service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:51.214704 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35066d2e-13cb-459f-81d6-a837ca27723f service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:51.214980 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35066d2e-13cb-459f-81d6-a837ca27723f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:51.224537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35066d2e-13cb-459f-81d6-a837ca27723f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:51.226109 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35066d2e-13cb-459f-81d6-a837ca27723f service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:51.236773 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 895/1780] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:15:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:15:51.628673 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675a521-ecfe-4e42-af74-b6ab568f5c85 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:51.630708 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-f675a521-ecfe-4e42-af74-b6ab568f5c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:51.646324 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675a521-ecfe-4e42-af74-b6ab568f5c85 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:51.653386 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ea5ea-bada-4c1c-9fad-2bf26cf60c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:51.655081 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ea5ea-bada-4c1c-9fad-2bf26cf60c24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:51.655392 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ea5ea-bada-4c1c-9fad-2bf26cf60c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:51.655711 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ea5ea-bada-4c1c-9fad-2bf26cf60c24 admin admin] PATH_INFO: `/v3/projects/675293a8cf7e48bc9adcb602b2a17d9f` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:51.657032 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ea5ea-bada-4c1c-9fad-2bf26cf60c24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=675293a8cf7e48bc9adcb602b2a17d9f)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:51.659031 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ea5ea-bada-4c1c-9fad-2bf26cf60c24 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:51.708341 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-a26ea5ea-bada-4c1c-9fad-2bf26cf60c24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '675293a8cf7e48bc9adcb602b2a17d9f', 'request_id': 'req-a26ea5ea-bada-4c1c-9fad-2bf26cf60c24'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:51.714994 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-a26ea5ea-bada-4c1c-9fad-2bf26cf60c24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '675293a8cf7e48bc9adcb602b2a17d9f', 'request_id': 'req-a26ea5ea-bada-4c1c-9fad-2bf26cf60c24'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:51.722988 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-a26ea5ea-bada-4c1c-9fad-2bf26cf60c24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '675293a8cf7e48bc9adcb602b2a17d9f', 'request_id': 'req-a26ea5ea-bada-4c1c-9fad-2bf26cf60c24'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:51.733996 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-a26ea5ea-bada-4c1c-9fad-2bf26cf60c24 admin admin] The token cache is being invalidate because project 675293a8cf7e48bc9adcb602b2a17d9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:15:51.734270 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-a26ea5ea-bada-4c1c-9fad-2bf26cf60c24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:51.734912 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-a26ea5ea-bada-4c1c-9fad-2bf26cf60c24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:51.736580 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 886/1781] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:15:51 2022] DELETE /identity/v3/projects/675293a8cf7e48bc9adcb602b2a17d9f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:51.740538 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35066d2e-13cb-459f-81d6-a837ca27723f service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:51.750527 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35066d2e-13cb-459f-81d6-a837ca27723f service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:51.806904 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b969545a-d636-4d7d-aa6a-0cc8adfffb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:51.807801 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b969545a-d636-4d7d-aa6a-0cc8adfffb10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:51.808083 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b969545a-d636-4d7d-aa6a-0cc8adfffb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:51.808359 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b969545a-d636-4d7d-aa6a-0cc8adfffb10 admin admin] PATH_INFO: `/v3/users/9ebfc8fcefdd4d7b96dfc3b0720bd1b5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:51.809561 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b969545a-d636-4d7d-aa6a-0cc8adfffb10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ebfc8fcefdd4d7b96dfc3b0720bd1b5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:51.812050 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b969545a-d636-4d7d-aa6a-0cc8adfffb10 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:51.856263 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b969545a-d636-4d7d-aa6a-0cc8adfffb10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ebfc8fcefdd4d7b96dfc3b0720bd1b5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:51.856895 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b969545a-d636-4d7d-aa6a-0cc8adfffb10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ebfc8fcefdd4d7b96dfc3b0720bd1b5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:51.868664 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b969545a-d636-4d7d-aa6a-0cc8adfffb10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ebfc8fcefdd4d7b96dfc3b0720bd1b5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:51.876153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b969545a-d636-4d7d-aa6a-0cc8adfffb10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ebfc8fcefdd4d7b96dfc3b0720bd1b5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:51.883878 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-b969545a-d636-4d7d-aa6a-0cc8adfffb10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ebfc8fcefdd4d7b96dfc3b0720bd1b5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:51.890832 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 896/1782] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:15:51 2022] DELETE /identity/v3/users/9ebfc8fcefdd4d7b96dfc3b0720bd1b5 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:52.176421 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ea5ea-bada-4c1c-9fad-2bf26cf60c24 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:52.178386 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-a26ea5ea-bada-4c1c-9fad-2bf26cf60c24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:52.200981 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ea5ea-bada-4c1c-9fad-2bf26cf60c24 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:52.252149 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:52.265803 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24d620-4176-4de6-a7e8-81b3b67ec10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:52.266915 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24d620-4176-4de6-a7e8-81b3b67ec10e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:52.267202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24d620-4176-4de6-a7e8-81b3b67ec10e admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:52.267501 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24d620-4176-4de6-a7e8-81b3b67ec10e admin admin] PATH_INFO: `/v3/projects/e45f472a49e3443eae6952b8037e5f5f` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:52.268768 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a24d620-4176-4de6-a7e8-81b3b67ec10e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e45f472a49e3443eae6952b8037e5f5f)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:52.270694 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a24d620-4176-4de6-a7e8-81b3b67ec10e admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:52.317981 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-5a24d620-4176-4de6-a7e8-81b3b67ec10e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e45f472a49e3443eae6952b8037e5f5f', 'request_id': 'req-5a24d620-4176-4de6-a7e8-81b3b67ec10e'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:52.323986 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-5a24d620-4176-4de6-a7e8-81b3b67ec10e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e45f472a49e3443eae6952b8037e5f5f', 'request_id': 'req-5a24d620-4176-4de6-a7e8-81b3b67ec10e'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:52.331320 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-5a24d620-4176-4de6-a7e8-81b3b67ec10e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e45f472a49e3443eae6952b8037e5f5f', 'request_id': 'req-5a24d620-4176-4de6-a7e8-81b3b67ec10e'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:52.341695 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-5a24d620-4176-4de6-a7e8-81b3b67ec10e admin admin] The token cache is being invalidate because project e45f472a49e3443eae6952b8037e5f5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:15:52.341977 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-5a24d620-4176-4de6-a7e8-81b3b67ec10e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:52.342617 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-5a24d620-4176-4de6-a7e8-81b3b67ec10e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:52.344199 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 887/1783] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:15:52 2022] DELETE /identity/v3/projects/e45f472a49e3443eae6952b8037e5f5f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:52.349249 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b160714-a3e9-4886-9132-1d0be7ffa491 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:52.349535 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b160714-a3e9-4886-9132-1d0be7ffa491 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:52.349831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b160714-a3e9-4886-9132-1d0be7ffa491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:52.370332 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-2b160714-a3e9-4886-9132-1d0be7ffa491 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:52.371461 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-2b160714-a3e9-4886-9132-1d0be7ffa491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:52.388729 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:52.435990 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 897/1784] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:15:52.443664 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d3aa5-de75-4834-8a69-6f13c5b7d218 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:52.443986 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d3aa5-de75-4834-8a69-6f13c5b7d218 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:52.444267 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d3aa5-de75-4834-8a69-6f13c5b7d218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:52.462557 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-841d3aa5-de75-4834-8a69-6f13c5b7d218 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:52.463819 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-841d3aa5-de75-4834-8a69-6f13c5b7d218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:52.471899 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 888/1785] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:52.476264 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b160714-a3e9-4886-9132-1d0be7ffa491 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:52.486734 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b160714-a3e9-4886-9132-1d0be7ffa491 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:52.491467 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481817b-36c7-49ab-a23f-1bcbeac4904c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:52.492194 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481817b-36c7-49ab-a23f-1bcbeac4904c admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:52.492482 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481817b-36c7-49ab-a23f-1bcbeac4904c admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:52.492755 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481817b-36c7-49ab-a23f-1bcbeac4904c admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:52.493484 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c481817b-36c7-49ab-a23f-1bcbeac4904c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:15:52.493864 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c481817b-36c7-49ab-a23f-1bcbeac4904c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:52.495584 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c481817b-36c7-49ab-a23f-1bcbeac4904c admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:52.508112 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 898/1786] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:15:52.512378 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d3aa5-de75-4834-8a69-6f13c5b7d218 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:52.522050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d3aa5-de75-4834-8a69-6f13c5b7d218 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:52.526752 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd7b50-1029-462d-9767-1513e7012740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:52.527491 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd7b50-1029-462d-9767-1513e7012740 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:52.527814 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd7b50-1029-462d-9767-1513e7012740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:52.528090 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd7b50-1029-462d-9767-1513e7012740 admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:52.529331 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd7b50-1029-462d-9767-1513e7012740 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:52.531256 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd7b50-1029-462d-9767-1513e7012740 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:52.564189 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 889/1787] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:15:52 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:52.565656 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481817b-36c7-49ab-a23f-1bcbeac4904c admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:52.577746 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481817b-36c7-49ab-a23f-1bcbeac4904c admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:52.642592 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f224c0c-8240-4516-8381-39879b26557b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:52.643748 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f224c0c-8240-4516-8381-39879b26557b admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:52.643748 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f224c0c-8240-4516-8381-39879b26557b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:52.643971 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f224c0c-8240-4516-8381-39879b26557b admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:52.644581 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f224c0c-8240-4516-8381-39879b26557b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:52.646613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f224c0c-8240-4516-8381-39879b26557b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:52.673777 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 899/1788] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:15:52 2022] POST /identity/v3/users => generated 294 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:52.676029 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd7b50-1029-462d-9767-1513e7012740 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:52.689348 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd7b50-1029-462d-9767-1513e7012740 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:52.694302 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913ef916-896b-4baf-b9bb-bd13d9580423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:52.695094 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ef916-896b-4baf-b9bb-bd13d9580423 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:52.695641 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ef916-896b-4baf-b9bb-bd13d9580423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:52.696053 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ef916-896b-4baf-b9bb-bd13d9580423 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:52.696868 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913ef916-896b-4baf-b9bb-bd13d9580423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:52.698574 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913ef916-896b-4baf-b9bb-bd13d9580423 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:52.712186 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 890/1789] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:15:52.716900 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f224c0c-8240-4516-8381-39879b26557b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:52.727165 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f224c0c-8240-4516-8381-39879b26557b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:52.732470 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f4dc1-a8c9-4e50-9f1a-104d0ede955d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:52.733136 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f4dc1-a8c9-4e50-9f1a-104d0ede955d admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:52.733478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f4dc1-a8c9-4e50-9f1a-104d0ede955d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:52.733681 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f4dc1-a8c9-4e50-9f1a-104d0ede955d admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:52.734413 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0f4dc1-a8c9-4e50-9f1a-104d0ede955d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:52.736179 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0f4dc1-a8c9-4e50-9f1a-104d0ede955d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:52.747605 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 900/1790] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:52.751754 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913ef916-896b-4baf-b9bb-bd13d9580423 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:52.761884 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913ef916-896b-4baf-b9bb-bd13d9580423 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:52.766410 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ca380-c02e-4a5b-91e6-b451056e0216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:52.767280 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ca380-c02e-4a5b-91e6-b451056e0216 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:52.767587 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ca380-c02e-4a5b-91e6-b451056e0216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:52.767863 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ca380-c02e-4a5b-91e6-b451056e0216 admin admin] PATH_INFO: `/v3/projects/414cd0d7f61f4c6f95bacaef6fcde66a/users/803a28dc36bf4726aff10500fa85a72f/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:52.782770 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8ca380-c02e-4a5b-91e6-b451056e0216 admin admin] RBAC: Authorizing `identity:create_grant(project_id=414cd0d7f61f4c6f95bacaef6fcde66a, user_id=803a28dc36bf4726aff10500fa85a72f, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:52.785442 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8ca380-c02e-4a5b-91e6-b451056e0216 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:52.799074 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 891/1791] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:15:52 2022] PUT /identity/v3/projects/414cd0d7f61f4c6f95bacaef6fcde66a/users/803a28dc36bf4726aff10500fa85a72f/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:52.809599 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d64f3-ae82-4542-add3-d726b7a71fb1 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:52.809894 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d64f3-ae82-4542-add3-d726b7a71fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:52.810177 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d64f3-ae82-4542-add3-d726b7a71fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:52.835989 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-f92d64f3-ae82-4542-add3-d726b7a71fb1 None None] MFA Rules not processed for user `803a28dc36bf4726aff10500fa85a72f`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:52.837220 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f92d64f3-ae82-4542-add3-d726b7a71fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:52.906458 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 901/1792] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:52.911014 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ca380-c02e-4a5b-91e6-b451056e0216 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:52.986659 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ca380-c02e-4a5b-91e6-b451056e0216 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:52.992229 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556328c4-f2cb-485a-86cd-bf8aa6ade282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:52.992949 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556328c4-f2cb-485a-86cd-bf8aa6ade282 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:52.993251 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556328c4-f2cb-485a-86cd-bf8aa6ade282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:52.993548 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556328c4-f2cb-485a-86cd-bf8aa6ade282 admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:52.994286 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556328c4-f2cb-485a-86cd-bf8aa6ade282 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:53.002593 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556328c4-f2cb-485a-86cd-bf8aa6ade282 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:53.022957 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:53.039066 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 892/1793] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:15:52 2022] POST /identity/v3/projects => generated 387 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:53.041934 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d64f3-ae82-4542-add3-d726b7a71fb1 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:53.055099 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d64f3-ae82-4542-add3-d726b7a71fb1 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:53.070031 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:53.185490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a63593-2878-492f-8a5a-14a8f3db42bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:53.186797 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a63593-2878-492f-8a5a-14a8f3db42bb admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:53.187183 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a63593-2878-492f-8a5a-14a8f3db42bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:53.187711 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a63593-2878-492f-8a5a-14a8f3db42bb admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:53.188420 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a63593-2878-492f-8a5a-14a8f3db42bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:53.190799 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a63593-2878-492f-8a5a-14a8f3db42bb admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:53.217274 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 902/1794] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:15:53 2022] POST /identity/v3/users => generated 300 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:53.225193 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556328c4-f2cb-485a-86cd-bf8aa6ade282 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:53.239452 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556328c4-f2cb-485a-86cd-bf8aa6ade282 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:53.251149 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759b1f94-f9c7-4b1d-bd8a-e8725e0f8a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:53.252035 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b1f94-f9c7-4b1d-bd8a-e8725e0f8a85 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:53.252422 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b1f94-f9c7-4b1d-bd8a-e8725e0f8a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:53.252820 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b1f94-f9c7-4b1d-bd8a-e8725e0f8a85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:53.253710 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759b1f94-f9c7-4b1d-bd8a-e8725e0f8a85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:53.255475 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759b1f94-f9c7-4b1d-bd8a-e8725e0f8a85 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:53.268883 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 893/1795] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:53.273268 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a63593-2878-492f-8a5a-14a8f3db42bb admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:53.285722 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a63593-2878-492f-8a5a-14a8f3db42bb admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:53.292034 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645670c2-3510-4af7-831f-b04887999199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:53.293102 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645670c2-3510-4af7-831f-b04887999199 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:53.293461 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645670c2-3510-4af7-831f-b04887999199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:53.293800 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645670c2-3510-4af7-831f-b04887999199 admin admin] PATH_INFO: `/v3/projects/4187d37b81974b94b513fdec9286d6c3/users/0c423be598ad4107aa424c102c2784bc/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:53.311723 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645670c2-3510-4af7-831f-b04887999199 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4187d37b81974b94b513fdec9286d6c3, user_id=0c423be598ad4107aa424c102c2784bc, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:53.319226 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645670c2-3510-4af7-831f-b04887999199 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:53.341467 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 903/1796] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:15:53 2022] PUT /identity/v3/projects/4187d37b81974b94b513fdec9286d6c3/users/0c423be598ad4107aa424c102c2784bc/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:15:53.345286 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759b1f94-f9c7-4b1d-bd8a-e8725e0f8a85 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:53.356697 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759b1f94-f9c7-4b1d-bd8a-e8725e0f8a85 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:53.420523 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95947e9b-2943-4b91-a787-588773f1749b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:53.421192 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95947e9b-2943-4b91-a787-588773f1749b admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:53.421475 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95947e9b-2943-4b91-a787-588773f1749b admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:53.421750 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95947e9b-2943-4b91-a787-588773f1749b admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:53.422470 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95947e9b-2943-4b91-a787-588773f1749b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:53.424118 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95947e9b-2943-4b91-a787-588773f1749b admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:53.434414 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 894/1797] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:53.439384 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645670c2-3510-4af7-831f-b04887999199 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:53.450982 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645670c2-3510-4af7-831f-b04887999199 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:53.455724 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3725de2a-dd61-442b-84ce-1eeed6699b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:53.456647 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725de2a-dd61-442b-84ce-1eeed6699b4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:53.456929 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725de2a-dd61-442b-84ce-1eeed6699b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:53.457204 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725de2a-dd61-442b-84ce-1eeed6699b4d admin admin] PATH_INFO: `/v3/projects/4187d37b81974b94b513fdec9286d6c3/users/0c423be598ad4107aa424c102c2784bc/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:53.459223 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3725de2a-dd61-442b-84ce-1eeed6699b4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4187d37b81974b94b513fdec9286d6c3, user_id=0c423be598ad4107aa424c102c2784bc, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:53.461895 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3725de2a-dd61-442b-84ce-1eeed6699b4d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:53.472659 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 904/1798] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:15:53 2022] PUT /identity/v3/projects/4187d37b81974b94b513fdec9286d6c3/users/0c423be598ad4107aa424c102c2784bc/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:53.483861 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95270a5c-3010-4e73-a3a2-fbf3ac523830 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:53.484154 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95270a5c-3010-4e73-a3a2-fbf3ac523830 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:53.484439 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95270a5c-3010-4e73-a3a2-fbf3ac523830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:53.526333 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-95270a5c-3010-4e73-a3a2-fbf3ac523830 None None] MFA Rules not processed for user `0c423be598ad4107aa424c102c2784bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:53.528327 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-95270a5c-3010-4e73-a3a2-fbf3ac523830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:53.631997 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 895/1799] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:53 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:53.650138 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3725de2a-dd61-442b-84ce-1eeed6699b4d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:53.650138 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3725de2a-dd61-442b-84ce-1eeed6699b4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:53.715820 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3725de2a-dd61-442b-84ce-1eeed6699b4d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:53.719363 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac229ef-10b0-4e06-b472-7438f495e3d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:53.720049 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac229ef-10b0-4e06-b472-7438f495e3d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:53.720333 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac229ef-10b0-4e06-b472-7438f495e3d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:53.720635 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac229ef-10b0-4e06-b472-7438f495e3d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:53.730514 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac229ef-10b0-4e06-b472-7438f495e3d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:53.732610 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac229ef-10b0-4e06-b472-7438f495e3d2 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:53.743327 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 905/1800] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:15:53 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:15:53.918354 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95270a5c-3010-4e73-a3a2-fbf3ac523830 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:54.024686 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95270a5c-3010-4e73-a3a2-fbf3ac523830 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:54.043727 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506c429-7322-4bbf-8086-1e0a8871ea60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:54.045609 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506c429-7322-4bbf-8086-1e0a8871ea60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:54.045991 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506c429-7322-4bbf-8086-1e0a8871ea60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:54.046357 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506c429-7322-4bbf-8086-1e0a8871ea60 admin admin] PATH_INFO: `/v3/users/803a28dc36bf4726aff10500fa85a72f` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:54.048105 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7506c429-7322-4bbf-8086-1e0a8871ea60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=803a28dc36bf4726aff10500fa85a72f)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:54.050104 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7506c429-7322-4bbf-8086-1e0a8871ea60 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:54.085193 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:54.136336 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-7506c429-7322-4bbf-8086-1e0a8871ea60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '803a28dc36bf4726aff10500fa85a72f'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:54.137154 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-7506c429-7322-4bbf-8086-1e0a8871ea60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '803a28dc36bf4726aff10500fa85a72f'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:54.156601 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-7506c429-7322-4bbf-8086-1e0a8871ea60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '803a28dc36bf4726aff10500fa85a72f'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:54.163501 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-7506c429-7322-4bbf-8086-1e0a8871ea60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '803a28dc36bf4726aff10500fa85a72f'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:54.164289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-7506c429-7322-4bbf-8086-1e0a8871ea60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '803a28dc36bf4726aff10500fa85a72f'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:54.179044 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 896/1801] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:15:53 2022] DELETE /identity/v3/users/803a28dc36bf4726aff10500fa85a72f => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:54.182386 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac229ef-10b0-4e06-b472-7438f495e3d2 service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:54.182386 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7ac229ef-10b0-4e06-b472-7438f495e3d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:54.183646 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bbe4cf-caab-40e6-9006-728ac348666f None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:54.183946 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bbe4cf-caab-40e6-9006-728ac348666f None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:54.184251 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bbe4cf-caab-40e6-9006-728ac348666f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:54.207171 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac229ef-10b0-4e06-b472-7438f495e3d2 service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:54.210024 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-38bbe4cf-caab-40e6-9006-728ac348666f None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:54.211313 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-38bbe4cf-caab-40e6-9006-728ac348666f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:54.256832 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3579bb-fb11-458d-a9af-d1ea04fc5cc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:54.257489 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3579bb-fb11-458d-a9af-d1ea04fc5cc5 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:54.257769 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3579bb-fb11-458d-a9af-d1ea04fc5cc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:54.258045 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3579bb-fb11-458d-a9af-d1ea04fc5cc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:54.260110 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-dd3579bb-fb11-458d-a9af-d1ea04fc5cc5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:54.272117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 897/1802] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:15:54.282172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bbe4cf-caab-40e6-9006-728ac348666f None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:54.283713 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-38bbe4cf-caab-40e6-9006-728ac348666f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:54.289228 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:54.339747 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3579bb-fb11-458d-a9af-d1ea04fc5cc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:54.342240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3579bb-fb11-458d-a9af-d1ea04fc5cc5 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:54.354201 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 906/1803] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:15:54 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:15:54.370387 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bbe4cf-caab-40e6-9006-728ac348666f None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:54.376002 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4781c1-acf9-4e49-935f-122110c8479d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:54.376675 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4781c1-acf9-4e49-935f-122110c8479d service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:54.376973 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4781c1-acf9-4e49-935f-122110c8479d service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:54.377313 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4781c1-acf9-4e49-935f-122110c8479d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:54.389017 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4781c1-acf9-4e49-935f-122110c8479d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:54.390486 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4781c1-acf9-4e49-935f-122110c8479d service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:54.404639 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 898/1804] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:15:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:15:54.863525 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3579bb-fb11-458d-a9af-d1ea04fc5cc5 service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:54.864977 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-dd3579bb-fb11-458d-a9af-d1ea04fc5cc5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:54.879521 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3579bb-fb11-458d-a9af-d1ea04fc5cc5 service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:54.884285 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10fa3f5-075c-4326-a615-7cdef5480472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:54.885148 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10fa3f5-075c-4326-a615-7cdef5480472 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:54.886261 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10fa3f5-075c-4326-a615-7cdef5480472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:54.886548 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10fa3f5-075c-4326-a615-7cdef5480472 admin admin] PATH_INFO: `/v3/projects/414cd0d7f61f4c6f95bacaef6fcde66a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:54.888512 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10fa3f5-075c-4326-a615-7cdef5480472 admin admin] RBAC: Authorizing `identity:delete_project(project_id=414cd0d7f61f4c6f95bacaef6fcde66a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:54.890417 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10fa3f5-075c-4326-a615-7cdef5480472 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:54.937090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c10fa3f5-075c-4326-a615-7cdef5480472 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '414cd0d7f61f4c6f95bacaef6fcde66a', 'request_id': 'req-c10fa3f5-075c-4326-a615-7cdef5480472'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:54.943592 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c10fa3f5-075c-4326-a615-7cdef5480472 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '414cd0d7f61f4c6f95bacaef6fcde66a', 'request_id': 'req-c10fa3f5-075c-4326-a615-7cdef5480472'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:54.951835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c10fa3f5-075c-4326-a615-7cdef5480472 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '414cd0d7f61f4c6f95bacaef6fcde66a', 'request_id': 'req-c10fa3f5-075c-4326-a615-7cdef5480472'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:54.962366 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c10fa3f5-075c-4326-a615-7cdef5480472 admin admin] The token cache is being invalidate because project 414cd0d7f61f4c6f95bacaef6fcde66a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:15:54.962650 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c10fa3f5-075c-4326-a615-7cdef5480472 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:54.963357 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c10fa3f5-075c-4326-a615-7cdef5480472 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:54.964962 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 907/1805] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:15:54 2022] DELETE /identity/v3/projects/414cd0d7f61f4c6f95bacaef6fcde66a => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:15:54.969055 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4781c1-acf9-4e49-935f-122110c8479d service neutron] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:54.978776 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4781c1-acf9-4e49-935f-122110c8479d service neutron] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:55.008426 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:55.037662 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c1834e-803d-4e68-9822-04a7f6c7dd50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:55.038549 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1834e-803d-4e68-9822-04a7f6c7dd50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:55.038853 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1834e-803d-4e68-9822-04a7f6c7dd50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:55.039096 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1834e-803d-4e68-9822-04a7f6c7dd50 admin admin] PATH_INFO: `/v3/users/0c423be598ad4107aa424c102c2784bc` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:55.040303 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c1834e-803d-4e68-9822-04a7f6c7dd50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c423be598ad4107aa424c102c2784bc)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:55.042285 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c1834e-803d-4e68-9822-04a7f6c7dd50 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:55.115938 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-74c1834e-803d-4e68-9822-04a7f6c7dd50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c423be598ad4107aa424c102c2784bc'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:55.116689 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-74c1834e-803d-4e68-9822-04a7f6c7dd50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c423be598ad4107aa424c102c2784bc'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:55.136134 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-74c1834e-803d-4e68-9822-04a7f6c7dd50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c423be598ad4107aa424c102c2784bc'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:55.144722 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-74c1834e-803d-4e68-9822-04a7f6c7dd50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c423be598ad4107aa424c102c2784bc'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:55.145484 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-74c1834e-803d-4e68-9822-04a7f6c7dd50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c423be598ad4107aa424c102c2784bc'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:55.160326 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 899/1806] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:15:54 2022] DELETE /identity/v3/users/0c423be598ad4107aa424c102c2784bc => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:55.467993 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10fa3f5-075c-4326-a615-7cdef5480472 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:55.469704 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c10fa3f5-075c-4326-a615-7cdef5480472 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:55.483516 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10fa3f5-075c-4326-a615-7cdef5480472 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:55.559171 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e68dc4-0f8e-4a5b-959f-e49975c09112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:55.560192 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e68dc4-0f8e-4a5b-959f-e49975c09112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:55.560455 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e68dc4-0f8e-4a5b-959f-e49975c09112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:55.560746 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e68dc4-0f8e-4a5b-959f-e49975c09112 admin admin] PATH_INFO: `/v3/projects/4187d37b81974b94b513fdec9286d6c3` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:55.561958 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e68dc4-0f8e-4a5b-959f-e49975c09112 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4187d37b81974b94b513fdec9286d6c3)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:55.563959 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e68dc4-0f8e-4a5b-959f-e49975c09112 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:55.613026 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:55.614328 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-26e68dc4-0f8e-4a5b-959f-e49975c09112 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4187d37b81974b94b513fdec9286d6c3', 'request_id': 'req-26e68dc4-0f8e-4a5b-959f-e49975c09112'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:55.623827 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-26e68dc4-0f8e-4a5b-959f-e49975c09112 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4187d37b81974b94b513fdec9286d6c3', 'request_id': 'req-26e68dc4-0f8e-4a5b-959f-e49975c09112'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:55.632253 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-26e68dc4-0f8e-4a5b-959f-e49975c09112 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4187d37b81974b94b513fdec9286d6c3', 'request_id': 'req-26e68dc4-0f8e-4a5b-959f-e49975c09112'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:55.642889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-26e68dc4-0f8e-4a5b-959f-e49975c09112 admin admin] The token cache is being invalidate because project 4187d37b81974b94b513fdec9286d6c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:15:55.643196 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-26e68dc4-0f8e-4a5b-959f-e49975c09112 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:55.643859 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-26e68dc4-0f8e-4a5b-959f-e49975c09112 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:55.645454 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 908/1807] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:15:55 2022] DELETE /identity/v3/projects/4187d37b81974b94b513fdec9286d6c3 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:15:55.650296 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc3ff7-c70a-4be6-9a8e-3f7653e04efe None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:55.650615 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc3ff7-c70a-4be6-9a8e-3f7653e04efe None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:55.650886 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc3ff7-c70a-4be6-9a8e-3f7653e04efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:55.675120 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-a8bc3ff7-c70a-4be6-9a8e-3f7653e04efe None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:55.676658 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-a8bc3ff7-c70a-4be6-9a8e-3f7653e04efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:55.739022 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 900/1808] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:55.746793 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c81eab-db0f-4e9f-87ad-4e077ef7c45b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:55.747114 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c81eab-db0f-4e9f-87ad-4e077ef7c45b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:55.747398 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c81eab-db0f-4e9f-87ad-4e077ef7c45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:55.768471 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-16c81eab-db0f-4e9f-87ad-4e077ef7c45b None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:55.769716 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-16c81eab-db0f-4e9f-87ad-4e077ef7c45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:55.779583 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 909/1809] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:55.783961 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc3ff7-c70a-4be6-9a8e-3f7653e04efe None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:55.794983 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc3ff7-c70a-4be6-9a8e-3f7653e04efe None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:55.800320 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657d0e18-6973-4c7b-a24d-90f067b6e547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:55.801152 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d0e18-6973-4c7b-a24d-90f067b6e547 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:55.810145 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d0e18-6973-4c7b-a24d-90f067b6e547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:55.810407 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d0e18-6973-4c7b-a24d-90f067b6e547 admin admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:55.820564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657d0e18-6973-4c7b-a24d-90f067b6e547 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:15:55.820564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657d0e18-6973-4c7b-a24d-90f067b6e547 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:55.820564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657d0e18-6973-4c7b-a24d-90f067b6e547 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:55.826901 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 901/1810] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:15:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:15:55.831082 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c81eab-db0f-4e9f-87ad-4e077ef7c45b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:55.845704 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c81eab-db0f-4e9f-87ad-4e077ef7c45b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:55.852565 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b7e17-0ec8-4b21-a599-fd2b8823db9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:55.853532 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b7e17-0ec8-4b21-a599-fd2b8823db9d admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:55.854000 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b7e17-0ec8-4b21-a599-fd2b8823db9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:55.854496 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b7e17-0ec8-4b21-a599-fd2b8823db9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:55.855402 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b7e17-0ec8-4b21-a599-fd2b8823db9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:55.859989 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b7e17-0ec8-4b21-a599-fd2b8823db9d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:55.891937 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 910/1811] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:15:55 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:55.896145 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657d0e18-6973-4c7b-a24d-90f067b6e547 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:55.907391 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657d0e18-6973-4c7b-a24d-90f067b6e547 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:55.949758 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:55.977837 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d935515b-c423-4a95-a3f2-dca907f1c087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:55.978579 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d935515b-c423-4a95-a3f2-dca907f1c087 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:55.978893 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d935515b-c423-4a95-a3f2-dca907f1c087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:55.979240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d935515b-c423-4a95-a3f2-dca907f1c087 admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:55.980088 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d935515b-c423-4a95-a3f2-dca907f1c087 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:55.982119 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d935515b-c423-4a95-a3f2-dca907f1c087 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:56.002448 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 902/1812] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:15:55 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:56.006541 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b7e17-0ec8-4b21-a599-fd2b8823db9d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:56.017355 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b7e17-0ec8-4b21-a599-fd2b8823db9d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:56.021956 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6354e08-ed31-4b7f-946d-4dc68feb2b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:56.022604 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6354e08-ed31-4b7f-946d-4dc68feb2b0a admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:56.022887 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6354e08-ed31-4b7f-946d-4dc68feb2b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:56.023166 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6354e08-ed31-4b7f-946d-4dc68feb2b0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:56.023917 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6354e08-ed31-4b7f-946d-4dc68feb2b0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:56.025507 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6354e08-ed31-4b7f-946d-4dc68feb2b0a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:56.036450 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 911/1813] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:56.040453 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d935515b-c423-4a95-a3f2-dca907f1c087 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:56.050481 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d935515b-c423-4a95-a3f2-dca907f1c087 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:56.055072 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b3e8ad-66e9-4589-96ae-f4d390ba9086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:56.055755 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3e8ad-66e9-4589-96ae-f4d390ba9086 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:56.056295 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3e8ad-66e9-4589-96ae-f4d390ba9086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:56.056573 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3e8ad-66e9-4589-96ae-f4d390ba9086 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:56.057303 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b3e8ad-66e9-4589-96ae-f4d390ba9086 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:56.058905 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b3e8ad-66e9-4589-96ae-f4d390ba9086 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:56.068874 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 903/1814] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:56.073011 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6354e08-ed31-4b7f-946d-4dc68feb2b0a admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:56.083494 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6354e08-ed31-4b7f-946d-4dc68feb2b0a admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:56.088130 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bfe860-a0a6-4ddd-a93b-affe173e7a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:56.089006 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bfe860-a0a6-4ddd-a93b-affe173e7a1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:56.089291 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bfe860-a0a6-4ddd-a93b-affe173e7a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:56.089571 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bfe860-a0a6-4ddd-a93b-affe173e7a1b admin admin] PATH_INFO: `/v3/projects/34894b6c4b5248efb095f466a4512fad/users/4546cda9370b4ab0b6d14abf0dcb5682/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:56.104802 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bfe860-a0a6-4ddd-a93b-affe173e7a1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=34894b6c4b5248efb095f466a4512fad, user_id=4546cda9370b4ab0b6d14abf0dcb5682, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:56.107520 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bfe860-a0a6-4ddd-a93b-affe173e7a1b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:56.121293 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 912/1815] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:15:56 2022] PUT /identity/v3/projects/34894b6c4b5248efb095f466a4512fad/users/4546cda9370b4ab0b6d14abf0dcb5682/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:15:56.131353 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5146162-bdf0-40d6-bc5e-63e164819c4c None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:56.131675 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5146162-bdf0-40d6-bc5e-63e164819c4c None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:56.131984 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5146162-bdf0-40d6-bc5e-63e164819c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:56.164629 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-d5146162-bdf0-40d6-bc5e-63e164819c4c None None] MFA Rules not processed for user `4546cda9370b4ab0b6d14abf0dcb5682`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:56.165865 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-d5146162-bdf0-40d6-bc5e-63e164819c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:56.238524 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 904/1816] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:56.250362 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bfe860-a0a6-4ddd-a93b-affe173e7a1b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:56.252282 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-17bfe860-a0a6-4ddd-a93b-affe173e7a1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:56.312057 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bfe860-a0a6-4ddd-a93b-affe173e7a1b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:56.315666 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da5b4e-cbaa-4ae2-a56d-4f412c7e3ab3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:56.316314 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da5b4e-cbaa-4ae2-a56d-4f412c7e3ab3 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:56.316595 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da5b4e-cbaa-4ae2-a56d-4f412c7e3ab3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:56.316870 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da5b4e-cbaa-4ae2-a56d-4f412c7e3ab3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:56.325804 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da5b4e-cbaa-4ae2-a56d-4f412c7e3ab3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:56.327258 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da5b4e-cbaa-4ae2-a56d-4f412c7e3ab3 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:56.336236 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 913/1817] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:15:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:15:56.355610 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5146162-bdf0-40d6-bc5e-63e164819c4c None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:56.422205 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5146162-bdf0-40d6-bc5e-63e164819c4c None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:56.426821 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624e9e4-430f-4357-96f1-7d7d5b47613b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:56.427757 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624e9e4-430f-4357-96f1-7d7d5b47613b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:56.428034 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624e9e4-430f-4357-96f1-7d7d5b47613b admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:56.428303 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624e9e4-430f-4357-96f1-7d7d5b47613b admin admin] PATH_INFO: `/v3/users/4546cda9370b4ab0b6d14abf0dcb5682` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:56.429574 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d624e9e4-430f-4357-96f1-7d7d5b47613b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4546cda9370b4ab0b6d14abf0dcb5682)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:56.431826 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d624e9e4-430f-4357-96f1-7d7d5b47613b admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:56.483221 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-d624e9e4-430f-4357-96f1-7d7d5b47613b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4546cda9370b4ab0b6d14abf0dcb5682'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:56.483933 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-d624e9e4-430f-4357-96f1-7d7d5b47613b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4546cda9370b4ab0b6d14abf0dcb5682'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:56.486279 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:56.501048 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-d624e9e4-430f-4357-96f1-7d7d5b47613b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4546cda9370b4ab0b6d14abf0dcb5682'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:56.507959 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-d624e9e4-430f-4357-96f1-7d7d5b47613b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4546cda9370b4ab0b6d14abf0dcb5682'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:56.508734 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-d624e9e4-430f-4357-96f1-7d7d5b47613b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4546cda9370b4ab0b6d14abf0dcb5682'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:56.523731 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 905/1818] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:15:56 2022] DELETE /identity/v3/users/4546cda9370b4ab0b6d14abf0dcb5682 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:56.528375 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e41ee-7f1e-45d4-a78b-a94c7d4d8e9c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:56.528844 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e41ee-7f1e-45d4-a78b-a94c7d4d8e9c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:56.528944 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e41ee-7f1e-45d4-a78b-a94c7d4d8e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:56.546743 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-361e41ee-7f1e-45d4-a78b-a94c7d4d8e9c None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:56.547995 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-361e41ee-7f1e-45d4-a78b-a94c7d4d8e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:56.611220 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 914/1819] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:56.621176 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624e9e4-430f-4357-96f1-7d7d5b47613b admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:56.622603 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-d624e9e4-430f-4357-96f1-7d7d5b47613b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:56.681149 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624e9e4-430f-4357-96f1-7d7d5b47613b admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:56.684793 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f58404f-68ef-474c-bbc2-84c194402b61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:56.685441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58404f-68ef-474c-bbc2-84c194402b61 service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:56.685727 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58404f-68ef-474c-bbc2-84c194402b61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:56.686005 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58404f-68ef-474c-bbc2-84c194402b61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:56.695786 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f58404f-68ef-474c-bbc2-84c194402b61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:56.697241 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f58404f-68ef-474c-bbc2-84c194402b61 service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:56.707883 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 906/1820] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:15:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:15:57.024664 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e41ee-7f1e-45d4-a78b-a94c7d4d8e9c None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:57.026248 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-361e41ee-7f1e-45d4-a78b-a94c7d4d8e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:57.039670 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e41ee-7f1e-45d4-a78b-a94c7d4d8e9c None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:57.046488 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c916105-f0b1-4391-8127-7945595b49cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:57.047456 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c916105-f0b1-4391-8127-7945595b49cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:57.047873 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c916105-f0b1-4391-8127-7945595b49cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:57.048217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c916105-f0b1-4391-8127-7945595b49cb admin admin] PATH_INFO: `/v3/projects/34894b6c4b5248efb095f466a4512fad` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:57.049550 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c916105-f0b1-4391-8127-7945595b49cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=34894b6c4b5248efb095f466a4512fad)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:57.052994 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c916105-f0b1-4391-8127-7945595b49cb admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:57.076086 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:57.114437 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2c916105-f0b1-4391-8127-7945595b49cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34894b6c4b5248efb095f466a4512fad', 'request_id': 'req-2c916105-f0b1-4391-8127-7945595b49cb'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:57.120501 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2c916105-f0b1-4391-8127-7945595b49cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34894b6c4b5248efb095f466a4512fad', 'request_id': 'req-2c916105-f0b1-4391-8127-7945595b49cb'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:57.127875 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2c916105-f0b1-4391-8127-7945595b49cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34894b6c4b5248efb095f466a4512fad', 'request_id': 'req-2c916105-f0b1-4391-8127-7945595b49cb'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:57.137994 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2c916105-f0b1-4391-8127-7945595b49cb admin admin] The token cache is being invalidate because project 34894b6c4b5248efb095f466a4512fad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:15:57.138280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2c916105-f0b1-4391-8127-7945595b49cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:57.138903 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2c916105-f0b1-4391-8127-7945595b49cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:57.140505 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 915/1821] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:15:57 2022] DELETE /identity/v3/projects/34894b6c4b5248efb095f466a4512fad => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:15:57.145644 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01b1b7-4d79-4e73-97ea-d1bd33099e56 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:57.146138 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01b1b7-4d79-4e73-97ea-d1bd33099e56 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:57.146231 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb01b1b7-4d79-4e73-97ea-d1bd33099e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:57.165949 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-fb01b1b7-4d79-4e73-97ea-d1bd33099e56 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:57.167175 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-fb01b1b7-4d79-4e73-97ea-d1bd33099e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:57.226370 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 907/1822] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:57.235744 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46245224-87c0-4030-997d-27846fef1737 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:57.236037 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46245224-87c0-4030-997d-27846fef1737 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:57.236318 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46245224-87c0-4030-997d-27846fef1737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:57.254468 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-46245224-87c0-4030-997d-27846fef1737 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:57.255728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-46245224-87c0-4030-997d-27846fef1737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:57.263668 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 916/1823] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:57.267961 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb01b1b7-4d79-4e73-97ea-d1bd33099e56 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:57.277661 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb01b1b7-4d79-4e73-97ea-d1bd33099e56 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:57.282756 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfaec2c-a5ae-48f0-a8e9-2720b78c4ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:57.283408 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfaec2c-a5ae-48f0-a8e9-2720b78c4ce6 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:57.283733 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfaec2c-a5ae-48f0-a8e9-2720b78c4ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:57.284012 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfaec2c-a5ae-48f0-a8e9-2720b78c4ce6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:57.284740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfaec2c-a5ae-48f0-a8e9-2720b78c4ce6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:15:57.285047 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfaec2c-a5ae-48f0-a8e9-2720b78c4ce6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:57.286652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfaec2c-a5ae-48f0-a8e9-2720b78c4ce6 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:57.299423 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 908/1824] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:15:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:15:57.303526 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46245224-87c0-4030-997d-27846fef1737 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:57.312978 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46245224-87c0-4030-997d-27846fef1737 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:57.317695 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527a55b0-f12b-4f61-ab96-05c5292f9299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:57.318342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a55b0-f12b-4f61-ab96-05c5292f9299 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:57.318628 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a55b0-f12b-4f61-ab96-05c5292f9299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:57.318902 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a55b0-f12b-4f61-ab96-05c5292f9299 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:57.319629 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527a55b0-f12b-4f61-ab96-05c5292f9299 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:57.321547 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527a55b0-f12b-4f61-ab96-05c5292f9299 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:57.347831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 917/1825] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:15:57 2022] POST /identity/v3/projects => generated 361 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:57.351963 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfaec2c-a5ae-48f0-a8e9-2720b78c4ce6 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:57.363433 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfaec2c-a5ae-48f0-a8e9-2720b78c4ce6 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:57.422261 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:57.429914 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be001b5-b81c-4502-9910-a18d2e861540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:57.430704 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be001b5-b81c-4502-9910-a18d2e861540 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:57.431488 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be001b5-b81c-4502-9910-a18d2e861540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:57.431973 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be001b5-b81c-4502-9910-a18d2e861540 admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:57.432875 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be001b5-b81c-4502-9910-a18d2e861540 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:57.434830 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be001b5-b81c-4502-9910-a18d2e861540 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:57.454747 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 909/1826] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:15:57 2022] POST /identity/v3/users => generated 281 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:57.458985 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527a55b0-f12b-4f61-ab96-05c5292f9299 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:57.470223 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527a55b0-f12b-4f61-ab96-05c5292f9299 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:57.475123 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a691b79-38d8-4fe6-9c96-394b0b6417d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:57.475831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a691b79-38d8-4fe6-9c96-394b0b6417d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:57.476128 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a691b79-38d8-4fe6-9c96-394b0b6417d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:57.476458 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a691b79-38d8-4fe6-9c96-394b0b6417d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:57.477153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a691b79-38d8-4fe6-9c96-394b0b6417d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:57.478769 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a691b79-38d8-4fe6-9c96-394b0b6417d7 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:57.492598 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 918/1827] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:57.493737 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be001b5-b81c-4502-9910-a18d2e861540 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:57.503771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be001b5-b81c-4502-9910-a18d2e861540 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:57.508340 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda62cd-401e-4f73-a817-40aeed875f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:57.508993 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda62cd-401e-4f73-a817-40aeed875f4e admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:57.509279 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda62cd-401e-4f73-a817-40aeed875f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:57.509554 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda62cd-401e-4f73-a817-40aeed875f4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:57.510283 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eda62cd-401e-4f73-a817-40aeed875f4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:57.511922 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eda62cd-401e-4f73-a817-40aeed875f4e admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:57.522593 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 910/1828] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:57.526849 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a691b79-38d8-4fe6-9c96-394b0b6417d7 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:57.537436 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a691b79-38d8-4fe6-9c96-394b0b6417d7 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:57.542092 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6759c3d-99ec-4a0f-94c3-2c893c4252c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:57.542976 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6759c3d-99ec-4a0f-94c3-2c893c4252c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:57.543254 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6759c3d-99ec-4a0f-94c3-2c893c4252c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:57.543528 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6759c3d-99ec-4a0f-94c3-2c893c4252c5 admin admin] PATH_INFO: `/v3/projects/75245483d0c94b809dca3fc475345e89/users/c0c2095570ee49639904b3cd57f047d5/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:57.559608 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6759c3d-99ec-4a0f-94c3-2c893c4252c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75245483d0c94b809dca3fc475345e89, user_id=c0c2095570ee49639904b3cd57f047d5, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:57.562302 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6759c3d-99ec-4a0f-94c3-2c893c4252c5 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:57.576350 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 919/1829] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:15:57 2022] PUT /identity/v3/projects/75245483d0c94b809dca3fc475345e89/users/c0c2095570ee49639904b3cd57f047d5/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:57.586606 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c485eea-e54f-4542-b987-6a8dd5b5fe64 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:57.586901 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c485eea-e54f-4542-b987-6a8dd5b5fe64 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:57.587181 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c485eea-e54f-4542-b987-6a8dd5b5fe64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:57.614717 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-3c485eea-e54f-4542-b987-6a8dd5b5fe64 None None] MFA Rules not processed for user `c0c2095570ee49639904b3cd57f047d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:57.615972 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-3c485eea-e54f-4542-b987-6a8dd5b5fe64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:57.681956 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 911/1830] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:57 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:57.714859 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6759c3d-99ec-4a0f-94c3-2c893c4252c5 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:57.717887 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a6759c3d-99ec-4a0f-94c3-2c893c4252c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:57.774977 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6759c3d-99ec-4a0f-94c3-2c893c4252c5 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:57.778530 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224fe413-3a81-4b12-923e-c24ed960c17f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:57.779170 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224fe413-3a81-4b12-923e-c24ed960c17f service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:57.779456 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224fe413-3a81-4b12-923e-c24ed960c17f service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:57.779765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224fe413-3a81-4b12-923e-c24ed960c17f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:57.788089 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224fe413-3a81-4b12-923e-c24ed960c17f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:57.789517 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224fe413-3a81-4b12-923e-c24ed960c17f service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:57.798584 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 920/1831] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:15:57 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:15:57.813008 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c485eea-e54f-4542-b987-6a8dd5b5fe64 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:57.875111 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c485eea-e54f-4542-b987-6a8dd5b5fe64 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:57.879880 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe892297-2c82-4245-b0cf-419f77219e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:57.880742 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe892297-2c82-4245-b0cf-419f77219e82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:57.881020 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe892297-2c82-4245-b0cf-419f77219e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:57.881290 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe892297-2c82-4245-b0cf-419f77219e82 admin admin] PATH_INFO: `/v3/users/c0c2095570ee49639904b3cd57f047d5` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:57.882479 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe892297-2c82-4245-b0cf-419f77219e82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0c2095570ee49639904b3cd57f047d5)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:57.884452 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe892297-2c82-4245-b0cf-419f77219e82 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:57.929944 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-fe892297-2c82-4245-b0cf-419f77219e82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0c2095570ee49639904b3cd57f047d5'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:57.930603 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-fe892297-2c82-4245-b0cf-419f77219e82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0c2095570ee49639904b3cd57f047d5'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:57.942843 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-fe892297-2c82-4245-b0cf-419f77219e82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0c2095570ee49639904b3cd57f047d5'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:57.944834 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:57.951458 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-fe892297-2c82-4245-b0cf-419f77219e82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0c2095570ee49639904b3cd57f047d5'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:57.952177 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-fe892297-2c82-4245-b0cf-419f77219e82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0c2095570ee49639904b3cd57f047d5'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:57.965157 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 912/1832] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:15:57 2022] DELETE /identity/v3/users/c0c2095570ee49639904b3cd57f047d5 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:57.969934 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d5801-8017-46fe-8884-1c0993478ec8 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:57.970249 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d5801-8017-46fe-8884-1c0993478ec8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:57.970529 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d5801-8017-46fe-8884-1c0993478ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:57.988398 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-4c4d5801-8017-46fe-8884-1c0993478ec8 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:57.989565 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4c4d5801-8017-46fe-8884-1c0993478ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:58.032780 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:58.067728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 921/1833] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:58.077894 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe892297-2c82-4245-b0cf-419f77219e82 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:58.079336 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-fe892297-2c82-4245-b0cf-419f77219e82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:58.154148 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe892297-2c82-4245-b0cf-419f77219e82 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:58.158074 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142fe8e9-6b6b-4b4e-ac0b-3bc83a8c0d9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:58.158745 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142fe8e9-6b6b-4b4e-ac0b-3bc83a8c0d9c service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:58.159178 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142fe8e9-6b6b-4b4e-ac0b-3bc83a8c0d9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:58.159525 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142fe8e9-6b6b-4b4e-ac0b-3bc83a8c0d9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:58.178951 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142fe8e9-6b6b-4b4e-ac0b-3bc83a8c0d9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:58.180488 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142fe8e9-6b6b-4b4e-ac0b-3bc83a8c0d9c service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:58.191853 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 913/1834] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:15:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:15:58.563918 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d5801-8017-46fe-8884-1c0993478ec8 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:58.565948 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4c4d5801-8017-46fe-8884-1c0993478ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:58.582301 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d5801-8017-46fe-8884-1c0993478ec8 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:58.588966 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875287d-8d1c-47d7-a686-ad45d493e4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:58.590034 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875287d-8d1c-47d7-a686-ad45d493e4f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:58.590579 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875287d-8d1c-47d7-a686-ad45d493e4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:58.591324 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875287d-8d1c-47d7-a686-ad45d493e4f4 admin admin] PATH_INFO: `/v3/projects/75245483d0c94b809dca3fc475345e89` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:58.592633 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1875287d-8d1c-47d7-a686-ad45d493e4f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75245483d0c94b809dca3fc475345e89)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:58.594652 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1875287d-8d1c-47d7-a686-ad45d493e4f4 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:58.648161 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1875287d-8d1c-47d7-a686-ad45d493e4f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75245483d0c94b809dca3fc475345e89', 'request_id': 'req-1875287d-8d1c-47d7-a686-ad45d493e4f4'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:58.655367 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1875287d-8d1c-47d7-a686-ad45d493e4f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75245483d0c94b809dca3fc475345e89', 'request_id': 'req-1875287d-8d1c-47d7-a686-ad45d493e4f4'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:58.667105 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1875287d-8d1c-47d7-a686-ad45d493e4f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75245483d0c94b809dca3fc475345e89', 'request_id': 'req-1875287d-8d1c-47d7-a686-ad45d493e4f4'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:58.681039 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1875287d-8d1c-47d7-a686-ad45d493e4f4 admin admin] The token cache is being invalidate because project 75245483d0c94b809dca3fc475345e89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:15:58.681318 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1875287d-8d1c-47d7-a686-ad45d493e4f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:58.682103 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1875287d-8d1c-47d7-a686-ad45d493e4f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:15:58.684342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 922/1835] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:15:58 2022] DELETE /identity/v3/projects/75245483d0c94b809dca3fc475345e89 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:58.688675 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c3864f-130d-4480-8554-d5855e849c88 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:58.688973 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c3864f-130d-4480-8554-d5855e849c88 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:58.689249 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c3864f-130d-4480-8554-d5855e849c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:58.708602 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-e8c3864f-130d-4480-8554-d5855e849c88 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:58.709995 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-e8c3864f-130d-4480-8554-d5855e849c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:58.791752 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 914/1836] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:15:58.792994 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f047281-6fcf-403a-bc05-24f2b511583c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:58.793289 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f047281-6fcf-403a-bc05-24f2b511583c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:58.793576 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f047281-6fcf-403a-bc05-24f2b511583c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:58.825874 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-5f047281-6fcf-403a-bc05-24f2b511583c None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:58.830113 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5f047281-6fcf-403a-bc05-24f2b511583c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:58.838480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 923/1837] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:15:58.844547 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c3864f-130d-4480-8554-d5855e849c88 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:58.859637 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c3864f-130d-4480-8554-d5855e849c88 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:58.865154 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07639f-5bda-4451-a723-21b5d915a686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:58.866384 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07639f-5bda-4451-a723-21b5d915a686 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:58.866703 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07639f-5bda-4451-a723-21b5d915a686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:58.867261 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07639f-5bda-4451-a723-21b5d915a686 admin admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:58.868350 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e07639f-5bda-4451-a723-21b5d915a686 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:15:58.868697 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e07639f-5bda-4451-a723-21b5d915a686 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:58.871198 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e07639f-5bda-4451-a723-21b5d915a686 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:58.894431 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 915/1838] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:15:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:15:58.895570 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f047281-6fcf-403a-bc05-24f2b511583c None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:58.909045 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f047281-6fcf-403a-bc05-24f2b511583c None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:58.925230 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac24095-7e86-413d-b693-aa5190945601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:58.925918 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac24095-7e86-413d-b693-aa5190945601 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:58.928346 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac24095-7e86-413d-b693-aa5190945601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:58.928669 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac24095-7e86-413d-b693-aa5190945601 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:58.929395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac24095-7e86-413d-b693-aa5190945601 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:58.934956 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac24095-7e86-413d-b693-aa5190945601 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:59.006624 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 924/1839] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:15:58 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:59.007923 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07639f-5bda-4451-a723-21b5d915a686 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:59.027698 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07639f-5bda-4451-a723-21b5d915a686 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:59.127416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf1a4a-33a8-4d7e-84f7-04a917e0b114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:59.128150 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf1a4a-33a8-4d7e-84f7-04a917e0b114 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:59.128435 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf1a4a-33a8-4d7e-84f7-04a917e0b114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:59.128711 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadf1a4a-33a8-4d7e-84f7-04a917e0b114 admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:59.129420 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadf1a4a-33a8-4d7e-84f7-04a917e0b114 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:59.134113 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadf1a4a-33a8-4d7e-84f7-04a917e0b114 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:59.161175 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 916/1840] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:15:59 2022] POST /identity/v3/users => generated 288 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:59.165304 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac24095-7e86-413d-b693-aa5190945601 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:59.178920 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac24095-7e86-413d-b693-aa5190945601 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:59.187474 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db29219b-fac5-4b66-8cab-22767a793fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:59.188276 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29219b-fac5-4b66-8cab-22767a793fa8 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:59.188621 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29219b-fac5-4b66-8cab-22767a793fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:59.188962 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29219b-fac5-4b66-8cab-22767a793fa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:59.189760 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db29219b-fac5-4b66-8cab-22767a793fa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:59.191452 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db29219b-fac5-4b66-8cab-22767a793fa8 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:59.207364 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 925/1841] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:15:59.208907 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf1a4a-33a8-4d7e-84f7-04a917e0b114 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:59.217101 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:59.221754 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadf1a4a-33a8-4d7e-84f7-04a917e0b114 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:59.229501 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2796486b-e193-4a84-bbb8-fe0aa2261402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:59.230361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796486b-e193-4a84-bbb8-fe0aa2261402 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:59.230822 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796486b-e193-4a84-bbb8-fe0aa2261402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:59.231415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796486b-e193-4a84-bbb8-fe0aa2261402 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:59.232719 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2796486b-e193-4a84-bbb8-fe0aa2261402 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:59.234729 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2796486b-e193-4a84-bbb8-fe0aa2261402 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:59.250482 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 917/1842] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:59.252203 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db29219b-fac5-4b66-8cab-22767a793fa8 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:59.263325 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db29219b-fac5-4b66-8cab-22767a793fa8 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:59.268801 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28650881-a5b8-4d33-8edb-9b69a3bba07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:59.269897 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28650881-a5b8-4d33-8edb-9b69a3bba07f admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:59.270156 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28650881-a5b8-4d33-8edb-9b69a3bba07f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:59.270419 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28650881-a5b8-4d33-8edb-9b69a3bba07f admin admin] PATH_INFO: `/v3/projects/9f9ea18eb83c4f50987875dcd5a904c2/users/635150515be44596ac42f5079e56f607/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:59.288596 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28650881-a5b8-4d33-8edb-9b69a3bba07f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f9ea18eb83c4f50987875dcd5a904c2, user_id=635150515be44596ac42f5079e56f607, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:59.291265 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28650881-a5b8-4d33-8edb-9b69a3bba07f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:59.313779 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 926/1843] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:15:59 2022] PUT /identity/v3/projects/9f9ea18eb83c4f50987875dcd5a904c2/users/635150515be44596ac42f5079e56f607/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:59.325706 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95770b3-0bf4-4855-bd68-36b67bc9b706 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:59.326338 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95770b3-0bf4-4855-bd68-36b67bc9b706 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:59.326766 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95770b3-0bf4-4855-bd68-36b67bc9b706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:59.363242 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-f95770b3-0bf4-4855-bd68-36b67bc9b706 None None] MFA Rules not processed for user `635150515be44596ac42f5079e56f607`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:15:59.365037 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-f95770b3-0bf4-4855-bd68-36b67bc9b706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:15:59.449337 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 918/1844] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:15:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:15:59.454544 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28650881-a5b8-4d33-8edb-9b69a3bba07f admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:59.548489 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:15:59.570081 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28650881-a5b8-4d33-8edb-9b69a3bba07f admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:59.578055 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd3a18-658c-4e55-ac03-883befb7c124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:59.579019 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd3a18-658c-4e55-ac03-883befb7c124 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:59.579456 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd3a18-658c-4e55-ac03-883befb7c124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:59.580487 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd3a18-658c-4e55-ac03-883befb7c124 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:59.581471 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cd3a18-658c-4e55-ac03-883befb7c124 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:59.583802 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cd3a18-658c-4e55-ac03-883befb7c124 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:59.627595 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 927/1845] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:15:59 2022] POST /identity/v3/projects => generated 375 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:59.632144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95770b3-0bf4-4855-bd68-36b67bc9b706 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:59.644983 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95770b3-0bf4-4855-bd68-36b67bc9b706 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:59.726080 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f22b6-a5a4-4ce9-b94b-c9f4ca88fdaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:59.726882 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f22b6-a5a4-4ce9-b94b-c9f4ca88fdaf admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:59.727178 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f22b6-a5a4-4ce9-b94b-c9f4ca88fdaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:59.727475 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f22b6-a5a4-4ce9-b94b-c9f4ca88fdaf admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:59.730525 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968f22b6-a5a4-4ce9-b94b-c9f4ca88fdaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:59.733176 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968f22b6-a5a4-4ce9-b94b-c9f4ca88fdaf admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:59.767384 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 919/1846] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:15:59 2022] POST /identity/v3/users => generated 294 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:15:59.772695 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd3a18-658c-4e55-ac03-883befb7c124 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:59.785499 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd3a18-658c-4e55-ac03-883befb7c124 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:59.790276 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731534ee-4310-462c-bf23-55ca65603662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:59.790955 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731534ee-4310-462c-bf23-55ca65603662 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:59.791362 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731534ee-4310-462c-bf23-55ca65603662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:59.791729 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731534ee-4310-462c-bf23-55ca65603662 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:59.792876 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731534ee-4310-462c-bf23-55ca65603662 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:59.795081 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731534ee-4310-462c-bf23-55ca65603662 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:59.812604 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 928/1847] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:15:59.818372 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f22b6-a5a4-4ce9-b94b-c9f4ca88fdaf admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:59.832509 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f22b6-a5a4-4ce9-b94b-c9f4ca88fdaf admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:59.839352 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4634c284-36dd-45d6-9a2f-74fbf2736d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:59.840543 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634c284-36dd-45d6-9a2f-74fbf2736d72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:59.840959 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634c284-36dd-45d6-9a2f-74fbf2736d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:59.841419 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4634c284-36dd-45d6-9a2f-74fbf2736d72 admin admin] PATH_INFO: `/v3/projects/71de5187a4e54eccbcadb425168911e8/users/227bd9bfac8f4d0ab0590d082641fea7/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:59.861133 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4634c284-36dd-45d6-9a2f-74fbf2736d72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71de5187a4e54eccbcadb425168911e8, user_id=227bd9bfac8f4d0ab0590d082641fea7, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:59.864801 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4634c284-36dd-45d6-9a2f-74fbf2736d72 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:15:59.882299 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 920/1848] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:15:59 2022] PUT /identity/v3/projects/71de5187a4e54eccbcadb425168911e8/users/227bd9bfac8f4d0ab0590d082641fea7/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:15:59.886592 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731534ee-4310-462c-bf23-55ca65603662 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:15:59.899341 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731534ee-4310-462c-bf23-55ca65603662 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:15:59.972310 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7629b103-ae51-4e88-a1b1-c545440b1362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:15:59.973377 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629b103-ae51-4e88-a1b1-c545440b1362 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:15:59.973825 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629b103-ae51-4e88-a1b1-c545440b1362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:15:59.974187 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629b103-ae51-4e88-a1b1-c545440b1362 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:15:59.974929 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7629b103-ae51-4e88-a1b1-c545440b1362 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:15:59.976787 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7629b103-ae51-4e88-a1b1-c545440b1362 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:00.000806 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 929/1849] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:16:00.004661 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4634c284-36dd-45d6-9a2f-74fbf2736d72 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:00.015687 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4634c284-36dd-45d6-9a2f-74fbf2736d72 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:00.020970 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa5494-f211-431e-8aef-173d3fef51d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:00.021874 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa5494-f211-431e-8aef-173d3fef51d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:00.022159 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa5494-f211-431e-8aef-173d3fef51d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:00.022433 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa5494-f211-431e-8aef-173d3fef51d4 admin admin] PATH_INFO: `/v3/projects/71de5187a4e54eccbcadb425168911e8/users/227bd9bfac8f4d0ab0590d082641fea7/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:00.024563 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfa5494-f211-431e-8aef-173d3fef51d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71de5187a4e54eccbcadb425168911e8, user_id=227bd9bfac8f4d0ab0590d082641fea7, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:00.027237 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfa5494-f211-431e-8aef-173d3fef51d4 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:00.040184 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 921/1850] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:16:00 2022] PUT /identity/v3/projects/71de5187a4e54eccbcadb425168911e8/users/227bd9bfac8f4d0ab0590d082641fea7/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:16:00.050929 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876fecb-9465-4e11-a6db-4c58326b83bc None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:00.051601 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876fecb-9465-4e11-a6db-4c58326b83bc None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:00.052019 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876fecb-9465-4e11-a6db-4c58326b83bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:00.088286 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-f876fecb-9465-4e11-a6db-4c58326b83bc None None] MFA Rules not processed for user `227bd9bfac8f4d0ab0590d082641fea7`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:16:00.089684 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f876fecb-9465-4e11-a6db-4c58326b83bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:00.218602 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 930/1851] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:16:00 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:16:00.240974 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa5494-f211-431e-8aef-173d3fef51d4 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:00.244102 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-5dfa5494-f211-431e-8aef-173d3fef51d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:00.366001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:00.410254 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa5494-f211-431e-8aef-173d3fef51d4 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:00.421503 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0b4102-1a71-4bf6-aa93-bb153baf793b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:00.421821 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b4102-1a71-4bf6-aa93-bb153baf793b service cinder] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:00.423434 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b4102-1a71-4bf6-aa93-bb153baf793b service cinder] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:00.423434 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b4102-1a71-4bf6-aa93-bb153baf793b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:00.529142 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0b4102-1a71-4bf6-aa93-bb153baf793b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:00.530701 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0b4102-1a71-4bf6-aa93-bb153baf793b service cinder] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:00.546881 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 922/1852] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:16:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:16:10.205119 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=75,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:10.536861 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=32 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:15.032289 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876fecb-9465-4e11-a6db-4c58326b83bc None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:15.098238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876fecb-9465-4e11-a6db-4c58326b83bc None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:15.103015 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ba6cab-29f0-4470-a7da-2c254c10da8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:15.103986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba6cab-29f0-4470-a7da-2c254c10da8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:15.104259 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba6cab-29f0-4470-a7da-2c254c10da8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:15.104527 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba6cab-29f0-4470-a7da-2c254c10da8f admin admin] PATH_INFO: `/v3/users/635150515be44596ac42f5079e56f607` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:15.106328 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ba6cab-29f0-4470-a7da-2c254c10da8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=635150515be44596ac42f5079e56f607)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:15.108310 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ba6cab-29f0-4470-a7da-2c254c10da8f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:15.160582 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-80ba6cab-29f0-4470-a7da-2c254c10da8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '635150515be44596ac42f5079e56f607'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:15.161388 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-80ba6cab-29f0-4470-a7da-2c254c10da8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '635150515be44596ac42f5079e56f607'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:15.174547 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-80ba6cab-29f0-4470-a7da-2c254c10da8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '635150515be44596ac42f5079e56f607'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:15.181541 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-80ba6cab-29f0-4470-a7da-2c254c10da8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '635150515be44596ac42f5079e56f607'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:15.189229 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-80ba6cab-29f0-4470-a7da-2c254c10da8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '635150515be44596ac42f5079e56f607'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:15.195269 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 931/1853] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:16:15 2022] DELETE /identity/v3/users/635150515be44596ac42f5079e56f607 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:16:15.200051 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53aecf8-98c7-42a9-aba5-5d4844d1d849 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:15.200565 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53aecf8-98c7-42a9-aba5-5d4844d1d849 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:15.200660 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53aecf8-98c7-42a9-aba5-5d4844d1d849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:15.220194 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-b53aecf8-98c7-42a9-aba5-5d4844d1d849 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:16:15.221428 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-b53aecf8-98c7-42a9-aba5-5d4844d1d849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:15.287149 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 923/1854] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:16:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:16:15.297766 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ba6cab-29f0-4470-a7da-2c254c10da8f admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:15.299207 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-80ba6cab-29f0-4470-a7da-2c254c10da8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:15.363589 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ba6cab-29f0-4470-a7da-2c254c10da8f admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:15.367372 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984d741b-d947-4f0f-9e6c-c0317993b19b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:15.368608 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d741b-d947-4f0f-9e6c-c0317993b19b service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:15.368608 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d741b-d947-4f0f-9e6c-c0317993b19b service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:15.369445 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d741b-d947-4f0f-9e6c-c0317993b19b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:15.381119 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984d741b-d947-4f0f-9e6c-c0317993b19b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:15.382573 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984d741b-d947-4f0f-9e6c-c0317993b19b service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:15.394831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 932/1855] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:16:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:16:16.276633 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53aecf8-98c7-42a9-aba5-5d4844d1d849 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:16.278129 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-b53aecf8-98c7-42a9-aba5-5d4844d1d849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:16.291785 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53aecf8-98c7-42a9-aba5-5d4844d1d849 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:16.305501 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af386a-718c-4058-b6f9-d96dfcfe8da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:16.306251 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af386a-718c-4058-b6f9-d96dfcfe8da5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:16.306545 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af386a-718c-4058-b6f9-d96dfcfe8da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:16.306843 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af386a-718c-4058-b6f9-d96dfcfe8da5 admin admin] PATH_INFO: `/v3/projects/9f9ea18eb83c4f50987875dcd5a904c2` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:16.308392 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42af386a-718c-4058-b6f9-d96dfcfe8da5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f9ea18eb83c4f50987875dcd5a904c2)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:16.311006 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42af386a-718c-4058-b6f9-d96dfcfe8da5 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:16.376436 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-42af386a-718c-4058-b6f9-d96dfcfe8da5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f9ea18eb83c4f50987875dcd5a904c2', 'request_id': 'req-42af386a-718c-4058-b6f9-d96dfcfe8da5'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:16.383286 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-42af386a-718c-4058-b6f9-d96dfcfe8da5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f9ea18eb83c4f50987875dcd5a904c2', 'request_id': 'req-42af386a-718c-4058-b6f9-d96dfcfe8da5'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:16.391256 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-42af386a-718c-4058-b6f9-d96dfcfe8da5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f9ea18eb83c4f50987875dcd5a904c2', 'request_id': 'req-42af386a-718c-4058-b6f9-d96dfcfe8da5'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:16.401881 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-42af386a-718c-4058-b6f9-d96dfcfe8da5 admin admin] The token cache is being invalidate because project 9f9ea18eb83c4f50987875dcd5a904c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:16:16.402262 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-42af386a-718c-4058-b6f9-d96dfcfe8da5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:16.403733 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-42af386a-718c-4058-b6f9-d96dfcfe8da5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:16.405337 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 924/1856] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:16:16 2022] DELETE /identity/v3/projects/9f9ea18eb83c4f50987875dcd5a904c2 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:16:16.409235 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984d741b-d947-4f0f-9e6c-c0317993b19b service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:16.419649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984d741b-d947-4f0f-9e6c-c0317993b19b service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:16.450041 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:16.497055 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029eeb8c-2f33-43dc-8bf5-52ea9d47cecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:16.498115 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029eeb8c-2f33-43dc-8bf5-52ea9d47cecd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:16.498462 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029eeb8c-2f33-43dc-8bf5-52ea9d47cecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:16.498805 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029eeb8c-2f33-43dc-8bf5-52ea9d47cecd admin admin] PATH_INFO: `/v3/users/227bd9bfac8f4d0ab0590d082641fea7` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:16.500160 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029eeb8c-2f33-43dc-8bf5-52ea9d47cecd admin admin] RBAC: Authorizing `identity:delete_user(user_id=227bd9bfac8f4d0ab0590d082641fea7)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:16.502265 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029eeb8c-2f33-43dc-8bf5-52ea9d47cecd admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:16.558877 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-029eeb8c-2f33-43dc-8bf5-52ea9d47cecd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '227bd9bfac8f4d0ab0590d082641fea7'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:16.559603 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-029eeb8c-2f33-43dc-8bf5-52ea9d47cecd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '227bd9bfac8f4d0ab0590d082641fea7'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:16.574937 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-029eeb8c-2f33-43dc-8bf5-52ea9d47cecd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '227bd9bfac8f4d0ab0590d082641fea7'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:16.583361 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-029eeb8c-2f33-43dc-8bf5-52ea9d47cecd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '227bd9bfac8f4d0ab0590d082641fea7'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:16.592684 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-029eeb8c-2f33-43dc-8bf5-52ea9d47cecd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '227bd9bfac8f4d0ab0590d082641fea7'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:16.599298 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 933/1857] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:16:16 2022] DELETE /identity/v3/users/227bd9bfac8f4d0ab0590d082641fea7 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:16:16.908068 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af386a-718c-4058-b6f9-d96dfcfe8da5 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:16.909882 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-42af386a-718c-4058-b6f9-d96dfcfe8da5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:16.929919 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af386a-718c-4058-b6f9-d96dfcfe8da5 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:17.006511 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea543f-071b-4290-aaca-0fb03e85fe32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:17.007459 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea543f-071b-4290-aaca-0fb03e85fe32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:17.007970 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea543f-071b-4290-aaca-0fb03e85fe32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:17.008462 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea543f-071b-4290-aaca-0fb03e85fe32 admin admin] PATH_INFO: `/v3/projects/71de5187a4e54eccbcadb425168911e8` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:17.009752 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea543f-071b-4290-aaca-0fb03e85fe32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71de5187a4e54eccbcadb425168911e8)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:17.012334 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea543f-071b-4290-aaca-0fb03e85fe32 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:17.050450 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:17.072303 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-2cea543f-071b-4290-aaca-0fb03e85fe32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71de5187a4e54eccbcadb425168911e8', 'request_id': 'req-2cea543f-071b-4290-aaca-0fb03e85fe32'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:17.079882 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-2cea543f-071b-4290-aaca-0fb03e85fe32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71de5187a4e54eccbcadb425168911e8', 'request_id': 'req-2cea543f-071b-4290-aaca-0fb03e85fe32'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:17.089477 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-2cea543f-071b-4290-aaca-0fb03e85fe32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71de5187a4e54eccbcadb425168911e8', 'request_id': 'req-2cea543f-071b-4290-aaca-0fb03e85fe32'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:17.102957 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-2cea543f-071b-4290-aaca-0fb03e85fe32 admin admin] The token cache is being invalidate because project 71de5187a4e54eccbcadb425168911e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:16:17.103253 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-2cea543f-071b-4290-aaca-0fb03e85fe32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:17.105009 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-2cea543f-071b-4290-aaca-0fb03e85fe32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:17.111953 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 925/1858] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:16:16 2022] DELETE /identity/v3/projects/71de5187a4e54eccbcadb425168911e8 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:16:17.113032 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb53da-c188-4e58-818e-d75a3f1901a8 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:17.113502 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb53da-c188-4e58-818e-d75a3f1901a8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:17.113855 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb53da-c188-4e58-818e-d75a3f1901a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:17.137776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-3dbb53da-c188-4e58-818e-d75a3f1901a8 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:16:17.138503 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3dbb53da-c188-4e58-818e-d75a3f1901a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:17.225256 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 934/1859] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:16:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:16:17.233376 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287c87b-7b39-4499-8a82-871068c6a750 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:17.233670 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287c87b-7b39-4499-8a82-871068c6a750 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:17.233952 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9287c87b-7b39-4499-8a82-871068c6a750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:17.259063 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-9287c87b-7b39-4499-8a82-871068c6a750 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:16:17.260435 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-9287c87b-7b39-4499-8a82-871068c6a750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:17.276248 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 926/1860] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:16:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:16:17.280801 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb53da-c188-4e58-818e-d75a3f1901a8 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:17.291457 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb53da-c188-4e58-818e-d75a3f1901a8 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:17.296904 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5b94c-a7c0-4c32-bf89-2dc0a81a6b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:17.297669 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5b94c-a7c0-4c32-bf89-2dc0a81a6b76 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:17.298019 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5b94c-a7c0-4c32-bf89-2dc0a81a6b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:17.298388 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5b94c-a7c0-4c32-bf89-2dc0a81a6b76 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:17.299204 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd5b94c-a7c0-4c32-bf89-2dc0a81a6b76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:16:17.299648 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd5b94c-a7c0-4c32-bf89-2dc0a81a6b76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:17.301340 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd5b94c-a7c0-4c32-bf89-2dc0a81a6b76 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:17.316060 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 935/1861] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:16:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:16:17.320359 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9287c87b-7b39-4499-8a82-871068c6a750 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:17.332491 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9287c87b-7b39-4499-8a82-871068c6a750 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:17.339093 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13535118-fb50-40c5-acdf-2a44a6078e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:17.340079 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13535118-fb50-40c5-acdf-2a44a6078e24 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:17.340506 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13535118-fb50-40c5-acdf-2a44a6078e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:17.340926 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13535118-fb50-40c5-acdf-2a44a6078e24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:17.342003 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13535118-fb50-40c5-acdf-2a44a6078e24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:17.344152 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13535118-fb50-40c5-acdf-2a44a6078e24 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:17.391591 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 927/1862] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:16:17 2022] POST /identity/v3/projects => generated 383 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:16:17.393251 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5b94c-a7c0-4c32-bf89-2dc0a81a6b76 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:17.407348 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5b94c-a7c0-4c32-bf89-2dc0a81a6b76 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:17.456756 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:17.515849 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16cdc27-6a79-4876-a811-855fddf40a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:17.516695 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16cdc27-6a79-4876-a811-855fddf40a20 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:17.516986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16cdc27-6a79-4876-a811-855fddf40a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:17.517306 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16cdc27-6a79-4876-a811-855fddf40a20 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:17.518117 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16cdc27-6a79-4876-a811-855fddf40a20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:17.520682 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16cdc27-6a79-4876-a811-855fddf40a20 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:17.553335 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 936/1863] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:16:17 2022] POST /identity/v3/users => generated 292 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:16:17.557495 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13535118-fb50-40c5-acdf-2a44a6078e24 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:17.569854 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13535118-fb50-40c5-acdf-2a44a6078e24 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:17.577112 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a2da8-1285-408f-8b6d-833100c5b9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:17.579865 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a2da8-1285-408f-8b6d-833100c5b9b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:17.579865 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a2da8-1285-408f-8b6d-833100c5b9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:17.579865 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a2da8-1285-408f-8b6d-833100c5b9b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:17.579865 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21a2da8-1285-408f-8b6d-833100c5b9b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:17.581514 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21a2da8-1285-408f-8b6d-833100c5b9b7 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:17.595669 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 928/1864] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:16:17.599001 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16cdc27-6a79-4876-a811-855fddf40a20 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:17.617509 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16cdc27-6a79-4876-a811-855fddf40a20 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:17.636110 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebd36cf-aae1-43e9-b45e-56dd16f7da76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:17.637483 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd36cf-aae1-43e9-b45e-56dd16f7da76 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:17.637872 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd36cf-aae1-43e9-b45e-56dd16f7da76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:17.638286 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd36cf-aae1-43e9-b45e-56dd16f7da76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:17.639155 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebd36cf-aae1-43e9-b45e-56dd16f7da76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:17.640896 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebd36cf-aae1-43e9-b45e-56dd16f7da76 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:17.657045 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 937/1865] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:16:17.661342 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a2da8-1285-408f-8b6d-833100c5b9b7 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:17.675398 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a2da8-1285-408f-8b6d-833100c5b9b7 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:17.696440 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632fe3c2-24b5-4589-8915-058b07396f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:17.698003 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632fe3c2-24b5-4589-8915-058b07396f61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:17.698536 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632fe3c2-24b5-4589-8915-058b07396f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:17.698979 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632fe3c2-24b5-4589-8915-058b07396f61 admin admin] PATH_INFO: `/v3/projects/07533f28418b4ceda75023f06edeff75/users/b6766ded508b4c74a66b8c75794e7dc1/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:17.729379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632fe3c2-24b5-4589-8915-058b07396f61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07533f28418b4ceda75023f06edeff75, user_id=b6766ded508b4c74a66b8c75794e7dc1, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:17.734213 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632fe3c2-24b5-4589-8915-058b07396f61 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:17.754776 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 929/1866] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:16:17 2022] PUT /identity/v3/projects/07533f28418b4ceda75023f06edeff75/users/b6766ded508b4c74a66b8c75794e7dc1/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:16:17.765720 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ddc995-bec7-4526-8714-631edeea174e None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:17.766233 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ddc995-bec7-4526-8714-631edeea174e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:17.766655 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ddc995-bec7-4526-8714-631edeea174e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:17.828764 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-e2ddc995-bec7-4526-8714-631edeea174e None None] MFA Rules not processed for user `b6766ded508b4c74a66b8c75794e7dc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:16:17.830934 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e2ddc995-bec7-4526-8714-631edeea174e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:17.846171 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b697b-0ebd-4944-b63a-68bb9899275a None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:17.846890 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b697b-0ebd-4944-b63a-68bb9899275a None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:17.847875 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b697b-0ebd-4944-b63a-68bb9899275a None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:17.849363 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 930/1867] 10.222.0.60 () {54 vars in 921 bytes} [Wed Jul 13 23:16:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:16:17.898635 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9678f-73ff-4712-bb00-31887923abfb None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:17.898925 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9678f-73ff-4712-bb00-31887923abfb None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:17.899206 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9678f-73ff-4712-bb00-31887923abfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:17.939509 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-71e9678f-73ff-4712-bb00-31887923abfb None None] MFA Rules not processed for user `4789384d158b4995bd30e1e37e30f5f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:16:17.943817 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-71e9678f-73ff-4712-bb00-31887923abfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:17.958316 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 938/1868] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:16:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:16:17.973760 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ddc995-bec7-4526-8714-631edeea174e None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:17.977152 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e2ddc995-bec7-4526-8714-631edeea174e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:18.076837 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 931/1869] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Jul 13 23:16:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:16:18.124547 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e9678f-73ff-4712-bb00-31887923abfb None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:18.126013 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-71e9678f-73ff-4712-bb00-31887923abfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:18.141072 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e9678f-73ff-4712-bb00-31887923abfb None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:18.148073 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ddc995-bec7-4526-8714-631edeea174e None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:18.148527 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40365026-edbb-40ae-a0dd-414dd52ce772 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:18.149490 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40365026-edbb-40ae-a0dd-414dd52ce772 service nova] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:18.149832 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40365026-edbb-40ae-a0dd-414dd52ce772 service nova] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:18.150318 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40365026-edbb-40ae-a0dd-414dd52ce772 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:18.156006 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d90a13a-c1c8-4d55-abe2-fe127f21f1e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:18.156799 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d90a13a-c1c8-4d55-abe2-fe127f21f1e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:18.157641 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d90a13a-c1c8-4d55-abe2-fe127f21f1e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:18.157641 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d90a13a-c1c8-4d55-abe2-fe127f21f1e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:18.162836 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40365026-edbb-40ae-a0dd-414dd52ce772 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:18.164695 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40365026-edbb-40ae-a0dd-414dd52ce772 service nova] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:18.166709 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d90a13a-c1c8-4d55-abe2-fe127f21f1e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:18.168907 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d90a13a-c1c8-4d55-abe2-fe127f21f1e3 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:18.179441 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 939/1870] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:16:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:16:18.183385 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 932/1871] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:16:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:16:22.728848 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f735b3a-b030-46dd-b522-c8c110c34488 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:22.729588 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f735b3a-b030-46dd-b522-c8c110c34488 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:22.729961 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f735b3a-b030-46dd-b522-c8c110c34488 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:22.732836 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 940/1872] 10.222.0.60 () {54 vars in 921 bytes} [Wed Jul 13 23:16:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:16:22.744068 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e166557-391e-4b18-a30e-63f4caa4516f None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:22.744545 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e166557-391e-4b18-a30e-63f4caa4516f None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:22.744902 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e166557-391e-4b18-a30e-63f4caa4516f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:22.768713 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-7e166557-391e-4b18-a30e-63f4caa4516f None None] MFA Rules not processed for user `4789384d158b4995bd30e1e37e30f5f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:16:22.770095 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-7e166557-391e-4b18-a30e-63f4caa4516f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:22.781822 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 933/1873] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Jul 13 23:16:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:16:22.810413 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f735b3a-b030-46dd-b522-c8c110c34488 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:22.812515 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3f735b3a-b030-46dd-b522-c8c110c34488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:22.832706 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f735b3a-b030-46dd-b522-c8c110c34488 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:22.838259 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18683f05-27e8-4111-a1b8-baa0bd271fe0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:22.838921 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18683f05-27e8-4111-a1b8-baa0bd271fe0 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:22.839207 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18683f05-27e8-4111-a1b8-baa0bd271fe0 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:22.839525 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18683f05-27e8-4111-a1b8-baa0bd271fe0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:22.849920 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18683f05-27e8-4111-a1b8-baa0bd271fe0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:22.851441 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18683f05-27e8-4111-a1b8-baa0bd271fe0 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:22.863570 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 941/1874] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:16:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:16:22.934542 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e166557-391e-4b18-a30e-63f4caa4516f None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:22.974171 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:23.031284 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e166557-391e-4b18-a30e-63f4caa4516f None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:23.036998 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae02eaf-0387-46fe-80a4-371899c233dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:23.038110 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae02eaf-0387-46fe-80a4-371899c233dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:23.038378 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae02eaf-0387-46fe-80a4-371899c233dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:23.038726 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae02eaf-0387-46fe-80a4-371899c233dd admin admin] PATH_INFO: `/v3/users/b6766ded508b4c74a66b8c75794e7dc1` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:23.040303 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae02eaf-0387-46fe-80a4-371899c233dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6766ded508b4c74a66b8c75794e7dc1)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:23.042254 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae02eaf-0387-46fe-80a4-371899c233dd admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:23.107048 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-9ae02eaf-0387-46fe-80a4-371899c233dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6766ded508b4c74a66b8c75794e7dc1'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:23.108475 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-9ae02eaf-0387-46fe-80a4-371899c233dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6766ded508b4c74a66b8c75794e7dc1'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:23.139285 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-9ae02eaf-0387-46fe-80a4-371899c233dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6766ded508b4c74a66b8c75794e7dc1'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:23.153398 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-9ae02eaf-0387-46fe-80a4-371899c233dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6766ded508b4c74a66b8c75794e7dc1'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:23.154081 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-9ae02eaf-0387-46fe-80a4-371899c233dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6766ded508b4c74a66b8c75794e7dc1'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:23.174999 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 934/1875] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:16:22 2022] DELETE /identity/v3/users/b6766ded508b4c74a66b8c75794e7dc1 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:16:23.179570 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c728ea-269b-4bbf-8e4d-779bdd34c8b8 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:23.180767 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c728ea-269b-4bbf-8e4d-779bdd34c8b8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:23.180767 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c728ea-269b-4bbf-8e4d-779bdd34c8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:23.202524 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-74c728ea-269b-4bbf-8e4d-779bdd34c8b8 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:16:23.205023 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-74c728ea-269b-4bbf-8e4d-779bdd34c8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:23.234537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:23.298106 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 942/1876] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:16:23.309021 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae02eaf-0387-46fe-80a4-371899c233dd admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:23.310534 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-9ae02eaf-0387-46fe-80a4-371899c233dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:23.371527 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae02eaf-0387-46fe-80a4-371899c233dd admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:23.375350 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369306a-84aa-497a-a41f-645e7b738647 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:23.376046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369306a-84aa-497a-a41f-645e7b738647 service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:23.376329 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369306a-84aa-497a-a41f-645e7b738647 service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:23.376607 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369306a-84aa-497a-a41f-645e7b738647 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:23.385994 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369306a-84aa-497a-a41f-645e7b738647 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:23.388186 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f369306a-84aa-497a-a41f-645e7b738647 service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:23.397745 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 935/1877] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:16:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:16:23.663903 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c728ea-269b-4bbf-8e4d-779bdd34c8b8 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:23.665422 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-74c728ea-269b-4bbf-8e4d-779bdd34c8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:23.692570 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c728ea-269b-4bbf-8e4d-779bdd34c8b8 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:23.697365 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f1370e-c6b8-43a4-8dc7-300424c974c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:23.698248 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1370e-c6b8-43a4-8dc7-300424c974c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:23.698538 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1370e-c6b8-43a4-8dc7-300424c974c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:23.698812 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1370e-c6b8-43a4-8dc7-300424c974c0 admin admin] PATH_INFO: `/v3/projects/07533f28418b4ceda75023f06edeff75` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:23.700064 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f1370e-c6b8-43a4-8dc7-300424c974c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07533f28418b4ceda75023f06edeff75)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:23.701966 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f1370e-c6b8-43a4-8dc7-300424c974c0 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:23.753263 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d0f1370e-c6b8-43a4-8dc7-300424c974c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07533f28418b4ceda75023f06edeff75', 'request_id': 'req-d0f1370e-c6b8-43a4-8dc7-300424c974c0'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:23.759568 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d0f1370e-c6b8-43a4-8dc7-300424c974c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07533f28418b4ceda75023f06edeff75', 'request_id': 'req-d0f1370e-c6b8-43a4-8dc7-300424c974c0'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:23.767163 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d0f1370e-c6b8-43a4-8dc7-300424c974c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07533f28418b4ceda75023f06edeff75', 'request_id': 'req-d0f1370e-c6b8-43a4-8dc7-300424c974c0'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:23.778373 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d0f1370e-c6b8-43a4-8dc7-300424c974c0 admin admin] The token cache is being invalidate because project 07533f28418b4ceda75023f06edeff75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:16:23.778668 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d0f1370e-c6b8-43a4-8dc7-300424c974c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:23.779314 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d0f1370e-c6b8-43a4-8dc7-300424c974c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:23.780916 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 943/1878] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:16:23 2022] DELETE /identity/v3/projects/07533f28418b4ceda75023f06edeff75 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:16:33.393467 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=69,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:33.775195 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:36.371558 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369306a-84aa-497a-a41f-645e7b738647 service neutron] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:36.373181 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-f369306a-84aa-497a-a41f-645e7b738647 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:36.389932 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369306a-84aa-497a-a41f-645e7b738647 service neutron] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:36.461117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd7a0a-2c5a-47b6-8b44-2a5ff30c31f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:36.462000 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd7a0a-2c5a-47b6-8b44-2a5ff30c31f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:36.462279 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd7a0a-2c5a-47b6-8b44-2a5ff30c31f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:36.462552 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd7a0a-2c5a-47b6-8b44-2a5ff30c31f9 admin admin] PATH_INFO: `/v3/users/1e12088c833f4df885e0e2ab4de2d757` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:36.463768 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cd7a0a-2c5a-47b6-8b44-2a5ff30c31f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e12088c833f4df885e0e2ab4de2d757)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:36.466364 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cd7a0a-2c5a-47b6-8b44-2a5ff30c31f9 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:36.518446 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-15cd7a0a-2c5a-47b6-8b44-2a5ff30c31f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e12088c833f4df885e0e2ab4de2d757'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:36.519234 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-15cd7a0a-2c5a-47b6-8b44-2a5ff30c31f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e12088c833f4df885e0e2ab4de2d757'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:36.533270 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-15cd7a0a-2c5a-47b6-8b44-2a5ff30c31f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e12088c833f4df885e0e2ab4de2d757'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:36.540207 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-15cd7a0a-2c5a-47b6-8b44-2a5ff30c31f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e12088c833f4df885e0e2ab4de2d757'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:36.540860 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-15cd7a0a-2c5a-47b6-8b44-2a5ff30c31f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e12088c833f4df885e0e2ab4de2d757'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:36.554029 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 936/1879] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:16:36 2022] DELETE /identity/v3/users/1e12088c833f4df885e0e2ab4de2d757 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:16:36.558866 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb88b0-09f3-4411-82a3-0a6927ac3996 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:36.559394 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb88b0-09f3-4411-82a3-0a6927ac3996 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:36.559480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb88b0-09f3-4411-82a3-0a6927ac3996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:36.579638 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-87cb88b0-09f3-4411-82a3-0a6927ac3996 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:16:36.580833 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-87cb88b0-09f3-4411-82a3-0a6927ac3996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:36.652496 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 944/1880] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:16:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:16:36.663606 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd7a0a-2c5a-47b6-8b44-2a5ff30c31f9 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:36.665275 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-15cd7a0a-2c5a-47b6-8b44-2a5ff30c31f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:36.740003 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd7a0a-2c5a-47b6-8b44-2a5ff30c31f9 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:36.743920 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67745ae0-68d2-409a-83bd-7f87d7e561c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:36.744594 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67745ae0-68d2-409a-83bd-7f87d7e561c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:36.746381 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67745ae0-68d2-409a-83bd-7f87d7e561c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:36.746672 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67745ae0-68d2-409a-83bd-7f87d7e561c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:36.759579 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67745ae0-68d2-409a-83bd-7f87d7e561c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:36.762236 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67745ae0-68d2-409a-83bd-7f87d7e561c8 service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:36.776724 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 937/1881] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:16:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:16:37.114796 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb88b0-09f3-4411-82a3-0a6927ac3996 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:37.116442 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-87cb88b0-09f3-4411-82a3-0a6927ac3996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:37.136989 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb88b0-09f3-4411-82a3-0a6927ac3996 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:37.142534 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5ac21-27ee-4307-a470-1de2837ddf9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:37.143525 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5ac21-27ee-4307-a470-1de2837ddf9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:37.143741 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5ac21-27ee-4307-a470-1de2837ddf9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:37.144022 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5ac21-27ee-4307-a470-1de2837ddf9c admin admin] PATH_INFO: `/v3/projects/729e4e5bcaae4189b6eabc30643ed12a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:37.145385 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e5ac21-27ee-4307-a470-1de2837ddf9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=729e4e5bcaae4189b6eabc30643ed12a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:37.147200 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e5ac21-27ee-4307-a470-1de2837ddf9c admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:37.202585 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-00e5ac21-27ee-4307-a470-1de2837ddf9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '729e4e5bcaae4189b6eabc30643ed12a', 'request_id': 'req-00e5ac21-27ee-4307-a470-1de2837ddf9c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:37.208858 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-00e5ac21-27ee-4307-a470-1de2837ddf9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '729e4e5bcaae4189b6eabc30643ed12a', 'request_id': 'req-00e5ac21-27ee-4307-a470-1de2837ddf9c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:37.216803 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-00e5ac21-27ee-4307-a470-1de2837ddf9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '729e4e5bcaae4189b6eabc30643ed12a', 'request_id': 'req-00e5ac21-27ee-4307-a470-1de2837ddf9c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:37.227340 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-00e5ac21-27ee-4307-a470-1de2837ddf9c admin admin] The token cache is being invalidate because project 729e4e5bcaae4189b6eabc30643ed12a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:16:37.227663 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-00e5ac21-27ee-4307-a470-1de2837ddf9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:37.228326 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-00e5ac21-27ee-4307-a470-1de2837ddf9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:37.230393 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 945/1882] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:16:37 2022] DELETE /identity/v3/projects/729e4e5bcaae4189b6eabc30643ed12a => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:16:37.234413 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67745ae0-68d2-409a-83bd-7f87d7e561c8 service neutron] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:37.244223 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67745ae0-68d2-409a-83bd-7f87d7e561c8 service neutron] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:37.264174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:37.311238 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba313c-1096-4a07-9bc4-fc6ef3de89c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:37.312206 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba313c-1096-4a07-9bc4-fc6ef3de89c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:37.312598 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba313c-1096-4a07-9bc4-fc6ef3de89c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:37.312882 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba313c-1096-4a07-9bc4-fc6ef3de89c6 admin admin] PATH_INFO: `/v3/users/04a550e3830e4655a724f952c2199103` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:37.314129 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba313c-1096-4a07-9bc4-fc6ef3de89c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04a550e3830e4655a724f952c2199103)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:37.316702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba313c-1096-4a07-9bc4-fc6ef3de89c6 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:37.369394 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-d8ba313c-1096-4a07-9bc4-fc6ef3de89c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04a550e3830e4655a724f952c2199103'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:37.370090 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-d8ba313c-1096-4a07-9bc4-fc6ef3de89c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04a550e3830e4655a724f952c2199103'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:37.383543 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-d8ba313c-1096-4a07-9bc4-fc6ef3de89c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04a550e3830e4655a724f952c2199103'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:37.390607 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-d8ba313c-1096-4a07-9bc4-fc6ef3de89c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04a550e3830e4655a724f952c2199103'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:37.391453 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-d8ba313c-1096-4a07-9bc4-fc6ef3de89c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04a550e3830e4655a724f952c2199103'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:37.406802 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 938/1883] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:16:37 2022] DELETE /identity/v3/users/04a550e3830e4655a724f952c2199103 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:16:37.695396 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5ac21-27ee-4307-a470-1de2837ddf9c admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:37.697527 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-00e5ac21-27ee-4307-a470-1de2837ddf9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:37.710932 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5ac21-27ee-4307-a470-1de2837ddf9c admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:37.779939 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8991c-ab03-44b9-9ec7-c277232c073b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:37.781326 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8991c-ab03-44b9-9ec7-c277232c073b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:37.781326 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8991c-ab03-44b9-9ec7-c277232c073b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:37.781626 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8991c-ab03-44b9-9ec7-c277232c073b admin admin] PATH_INFO: `/v3/projects/a61a6cf8b59a42a2880d49cb0ccc3432` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:37.783011 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8991c-ab03-44b9-9ec7-c277232c073b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a61a6cf8b59a42a2880d49cb0ccc3432)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:37.785044 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8991c-ab03-44b9-9ec7-c277232c073b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:37.834205 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:37.862747 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-84e8991c-ab03-44b9-9ec7-c277232c073b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a61a6cf8b59a42a2880d49cb0ccc3432', 'request_id': 'req-84e8991c-ab03-44b9-9ec7-c277232c073b'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:37.871693 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-84e8991c-ab03-44b9-9ec7-c277232c073b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a61a6cf8b59a42a2880d49cb0ccc3432', 'request_id': 'req-84e8991c-ab03-44b9-9ec7-c277232c073b'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:37.882552 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-84e8991c-ab03-44b9-9ec7-c277232c073b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a61a6cf8b59a42a2880d49cb0ccc3432', 'request_id': 'req-84e8991c-ab03-44b9-9ec7-c277232c073b'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:37.905982 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-84e8991c-ab03-44b9-9ec7-c277232c073b admin admin] The token cache is being invalidate because project a61a6cf8b59a42a2880d49cb0ccc3432 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:16:37.906349 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-84e8991c-ab03-44b9-9ec7-c277232c073b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:37.907810 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-84e8991c-ab03-44b9-9ec7-c277232c073b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:37.910552 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 946/1884] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:16:37 2022] DELETE /identity/v3/projects/a61a6cf8b59a42a2880d49cb0ccc3432 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:16:37.916229 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fbd03e-7c4f-4686-9d02-99644f5b3268 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:37.916544 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fbd03e-7c4f-4686-9d02-99644f5b3268 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:37.916831 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fbd03e-7c4f-4686-9d02-99644f5b3268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:37.946832 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-71fbd03e-7c4f-4686-9d02-99644f5b3268 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:16:37.948092 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-71fbd03e-7c4f-4686-9d02-99644f5b3268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:38.031931 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 939/1885] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:16:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:16:38.039093 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3cd904-9dcc-4443-ac35-217837148307 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:38.039402 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3cd904-9dcc-4443-ac35-217837148307 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:38.039723 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3cd904-9dcc-4443-ac35-217837148307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:38.059272 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-2e3cd904-9dcc-4443-ac35-217837148307 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:16:38.060552 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-2e3cd904-9dcc-4443-ac35-217837148307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:38.077269 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 947/1886] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:16:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:16:38.083904 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fbd03e-7c4f-4686-9d02-99644f5b3268 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:38.095798 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fbd03e-7c4f-4686-9d02-99644f5b3268 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:38.100974 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d593e53-eded-42bd-9a07-f9a44237a675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:38.101608 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d593e53-eded-42bd-9a07-f9a44237a675 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:38.101891 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d593e53-eded-42bd-9a07-f9a44237a675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:38.102169 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d593e53-eded-42bd-9a07-f9a44237a675 admin admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:38.102904 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d593e53-eded-42bd-9a07-f9a44237a675 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:16:38.103221 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d593e53-eded-42bd-9a07-f9a44237a675 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:38.104925 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d593e53-eded-42bd-9a07-f9a44237a675 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:38.117230 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 940/1887] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:16:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:16:38.122212 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3cd904-9dcc-4443-ac35-217837148307 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:38.133443 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3cd904-9dcc-4443-ac35-217837148307 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:38.140117 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa589c49-fc55-445d-8681-522efd95146e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:38.141561 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa589c49-fc55-445d-8681-522efd95146e admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:38.141972 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa589c49-fc55-445d-8681-522efd95146e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:38.142252 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa589c49-fc55-445d-8681-522efd95146e admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:38.142954 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa589c49-fc55-445d-8681-522efd95146e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:38.145020 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa589c49-fc55-445d-8681-522efd95146e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:38.175227 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 948/1888] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:16:38 2022] POST /identity/v3/projects => generated 387 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:16:38.179582 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d593e53-eded-42bd-9a07-f9a44237a675 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:38.191846 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d593e53-eded-42bd-9a07-f9a44237a675 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:38.216508 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa589c49-fc55-445d-8681-522efd95146e admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:38.218480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-aa589c49-fc55-445d-8681-522efd95146e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:38.229880 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:38.273933 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b31d2-c47d-45e8-90f9-abd6dc8b1ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:38.274633 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b31d2-c47d-45e8-90f9-abd6dc8b1ff9 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:38.274918 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b31d2-c47d-45e8-90f9-abd6dc8b1ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:38.275195 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b31d2-c47d-45e8-90f9-abd6dc8b1ff9 admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:38.275939 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81b31d2-c47d-45e8-90f9-abd6dc8b1ff9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:38.277972 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81b31d2-c47d-45e8-90f9-abd6dc8b1ff9 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:38.299840 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 941/1889] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:16:38 2022] POST /identity/v3/users => generated 294 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:16:38.304306 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b31d2-c47d-45e8-90f9-abd6dc8b1ff9 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:38.310122 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa589c49-fc55-445d-8681-522efd95146e admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:38.315775 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b31d2-c47d-45e8-90f9-abd6dc8b1ff9 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:38.316017 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41d66f-30c2-42f8-a0b6-c1a54edac3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:38.316882 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41d66f-30c2-42f8-a0b6-c1a54edac3ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:38.317168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41d66f-30c2-42f8-a0b6-c1a54edac3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:38.317488 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41d66f-30c2-42f8-a0b6-c1a54edac3ef admin admin] PATH_INFO: `/v3/users/f624cdb8f49f4eaaaea665afe0a72602` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:38.318835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e41d66f-30c2-42f8-a0b6-c1a54edac3ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=f624cdb8f49f4eaaaea665afe0a72602)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:38.320827 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e41d66f-30c2-42f8-a0b6-c1a54edac3ef admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:38.321452 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9231c-4f57-444f-855a-46a4769e29cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:38.322792 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9231c-4f57-444f-855a-46a4769e29cb admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:38.323077 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9231c-4f57-444f-855a-46a4769e29cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:38.323934 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9231c-4f57-444f-855a-46a4769e29cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:38.324731 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c9231c-4f57-444f-855a-46a4769e29cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:38.326898 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c9231c-4f57-444f-855a-46a4769e29cb admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:38.342574 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 942/1890] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:16:38 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:16:38.346704 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9231c-4f57-444f-855a-46a4769e29cb admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:38.367054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9231c-4f57-444f-855a-46a4769e29cb admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:38.374491 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbbac53-682e-462d-b57f-22aca4556a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:38.375502 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbac53-682e-462d-b57f-22aca4556a05 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:38.375815 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbac53-682e-462d-b57f-22aca4556a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:38.376426 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbac53-682e-462d-b57f-22aca4556a05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:38.377442 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbbac53-682e-462d-b57f-22aca4556a05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:38.379943 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbbac53-682e-462d-b57f-22aca4556a05 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:38.385419 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0e41d66f-30c2-42f8-a0b6-c1a54edac3ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f624cdb8f49f4eaaaea665afe0a72602'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:38.386999 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0e41d66f-30c2-42f8-a0b6-c1a54edac3ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f624cdb8f49f4eaaaea665afe0a72602'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:38.391861 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 943/1891] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:16:38 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:16:38.396146 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbbac53-682e-462d-b57f-22aca4556a05 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:38.398496 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-9fbbac53-682e-462d-b57f-22aca4556a05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:38.401396 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0e41d66f-30c2-42f8-a0b6-c1a54edac3ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f624cdb8f49f4eaaaea665afe0a72602'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:38.409801 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0e41d66f-30c2-42f8-a0b6-c1a54edac3ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f624cdb8f49f4eaaaea665afe0a72602'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:38.411900 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbbac53-682e-462d-b57f-22aca4556a05 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:38.417488 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d90e35-f2a7-4e95-8169-b48d4665c20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:38.418014 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0e41d66f-30c2-42f8-a0b6-c1a54edac3ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f624cdb8f49f4eaaaea665afe0a72602'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:38.418704 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d90e35-f2a7-4e95-8169-b48d4665c20f admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:38.418990 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d90e35-f2a7-4e95-8169-b48d4665c20f admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:38.419397 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d90e35-f2a7-4e95-8169-b48d4665c20f admin admin] PATH_INFO: `/v3/projects/96d5e99d8e394573b45abbb022e4d166/users/8630010672024c7eac5c38f1828e74cb/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:38.424645 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 949/1892] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:16:38 2022] DELETE /identity/v3/users/f624cdb8f49f4eaaaea665afe0a72602 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:16:38.429013 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db0a72-9880-41cd-b75b-e05f224cfb60 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:38.429305 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db0a72-9880-41cd-b75b-e05f224cfb60 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:38.429860 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db0a72-9880-41cd-b75b-e05f224cfb60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:38.436846 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d90e35-f2a7-4e95-8169-b48d4665c20f admin admin] RBAC: Authorizing `identity:create_grant(project_id=96d5e99d8e394573b45abbb022e4d166, user_id=8630010672024c7eac5c38f1828e74cb, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:38.437002 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:38.439757 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d90e35-f2a7-4e95-8169-b48d4665c20f admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:38.449954 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-53db0a72-9880-41cd-b75b-e05f224cfb60 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:16:38.451851 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-53db0a72-9880-41cd-b75b-e05f224cfb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:38.454902 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 944/1893] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:16:38 2022] PUT /identity/v3/projects/96d5e99d8e394573b45abbb022e4d166/users/8630010672024c7eac5c38f1828e74cb/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:16:38.465403 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b347d8e-808a-408e-b769-754bbcf88668 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:38.465689 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b347d8e-808a-408e-b769-754bbcf88668 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:38.465971 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b347d8e-808a-408e-b769-754bbcf88668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:38.493642 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-4b347d8e-808a-408e-b769-754bbcf88668 None None] MFA Rules not processed for user `8630010672024c7eac5c38f1828e74cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:16:38.494942 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-4b347d8e-808a-408e-b769-754bbcf88668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:38.517113 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 950/1894] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:16:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:16:38.529512 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53db0a72-9880-41cd-b75b-e05f224cfb60 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:38.533084 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-53db0a72-9880-41cd-b75b-e05f224cfb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:38.590710 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 945/1895] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:16:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:16:38.595769 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b347d8e-808a-408e-b769-754bbcf88668 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:38.607944 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b347d8e-808a-408e-b769-754bbcf88668 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:38.609784 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53db0a72-9880-41cd-b75b-e05f224cfb60 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:38.613357 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f4bbe1-fbc0-4127-9174-4c214de1edc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:38.614004 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ed1dfd-c0c4-40fb-849f-39a413aa4261 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:38.614153 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4bbe1-fbc0-4127-9174-4c214de1edc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:38.614296 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4bbe1-fbc0-4127-9174-4c214de1edc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:38.614574 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f4bbe1-fbc0-4127-9174-4c214de1edc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:38.615231 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed1dfd-c0c4-40fb-849f-39a413aa4261 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:38.615359 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f4bbe1-fbc0-4127-9174-4c214de1edc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:38.615802 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed1dfd-c0c4-40fb-849f-39a413aa4261 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:38.616222 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed1dfd-c0c4-40fb-849f-39a413aa4261 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:38.617449 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f4bbe1-fbc0-4127-9174-4c214de1edc2 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:38.630170 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ed1dfd-c0c4-40fb-849f-39a413aa4261 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:38.632227 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ed1dfd-c0c4-40fb-849f-39a413aa4261 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:38.646223 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 951/1896] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:16:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:16:38.648712 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 946/1897] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:16:38 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:16:38.652590 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ed1dfd-c0c4-40fb-849f-39a413aa4261 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:38.662842 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ed1dfd-c0c4-40fb-849f-39a413aa4261 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:38.750117 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fa3c89-f461-480b-bfd5-c39d20cb7892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:38.752599 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa3c89-f461-480b-bfd5-c39d20cb7892 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:38.753069 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa3c89-f461-480b-bfd5-c39d20cb7892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:38.753390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa3c89-f461-480b-bfd5-c39d20cb7892 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:38.758676 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fa3c89-f461-480b-bfd5-c39d20cb7892 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:38.763379 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fa3c89-f461-480b-bfd5-c39d20cb7892 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:38.818314 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 952/1898] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:16:38 2022] POST /identity/v3/users => generated 299 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:16:38.822770 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f4bbe1-fbc0-4127-9174-4c214de1edc2 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:38.838942 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f4bbe1-fbc0-4127-9174-4c214de1edc2 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:38.848358 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4205e2c-2261-495e-a822-ef86be9a1d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:38.849334 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4205e2c-2261-495e-a822-ef86be9a1d41 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:38.849764 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4205e2c-2261-495e-a822-ef86be9a1d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:38.850203 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4205e2c-2261-495e-a822-ef86be9a1d41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:38.851122 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4205e2c-2261-495e-a822-ef86be9a1d41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:38.853270 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4205e2c-2261-495e-a822-ef86be9a1d41 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:38.870867 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 947/1899] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:16:38 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:16:38.875441 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fa3c89-f461-480b-bfd5-c39d20cb7892 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:38.907038 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fa3c89-f461-480b-bfd5-c39d20cb7892 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:38.919569 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15772b16-c9e7-4a1f-b757-480d08c8b1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:38.920724 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15772b16-c9e7-4a1f-b757-480d08c8b1ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:38.921030 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15772b16-c9e7-4a1f-b757-480d08c8b1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:38.921030 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15772b16-c9e7-4a1f-b757-480d08c8b1ef admin admin] PATH_INFO: `/v3/projects/41eacb06bb8549698c2f648e57e1ca0b/users/ae2b410ff4b645beb42fde319048c68a/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:38.946207 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15772b16-c9e7-4a1f-b757-480d08c8b1ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=41eacb06bb8549698c2f648e57e1ca0b, user_id=ae2b410ff4b645beb42fde319048c68a, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:38.950290 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15772b16-c9e7-4a1f-b757-480d08c8b1ef admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:38.974057 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 953/1900] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:16:38 2022] PUT /identity/v3/projects/41eacb06bb8549698c2f648e57e1ca0b/users/ae2b410ff4b645beb42fde319048c68a/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:16:38.978450 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4205e2c-2261-495e-a822-ef86be9a1d41 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:38.994482 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4205e2c-2261-495e-a822-ef86be9a1d41 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:39.029120 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:39.066920 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86ae067-00af-4326-953f-7e9219b33ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:39.067617 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ae067-00af-4326-953f-7e9219b33ff6 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:39.067865 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ae067-00af-4326-953f-7e9219b33ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:39.068146 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ae067-00af-4326-953f-7e9219b33ff6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:39.068974 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86ae067-00af-4326-953f-7e9219b33ff6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:39.072049 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86ae067-00af-4326-953f-7e9219b33ff6 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:39.087364 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 948/1901] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:16:38 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:16:39.092572 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15772b16-c9e7-4a1f-b757-480d08c8b1ef admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:39.116076 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15772b16-c9e7-4a1f-b757-480d08c8b1ef admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:39.122236 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4194ba6e-ac7e-410f-8a60-93bbb8462e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:39.123255 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194ba6e-ac7e-410f-8a60-93bbb8462e27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:39.123557 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194ba6e-ac7e-410f-8a60-93bbb8462e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:39.123861 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194ba6e-ac7e-410f-8a60-93bbb8462e27 admin admin] PATH_INFO: `/v3/projects/41eacb06bb8549698c2f648e57e1ca0b/users/ae2b410ff4b645beb42fde319048c68a/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:39.126498 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4194ba6e-ac7e-410f-8a60-93bbb8462e27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41eacb06bb8549698c2f648e57e1ca0b, user_id=ae2b410ff4b645beb42fde319048c68a, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:39.129380 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4194ba6e-ac7e-410f-8a60-93bbb8462e27 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:39.138207 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:39.156397 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 954/1902] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:16:39 2022] PUT /identity/v3/projects/41eacb06bb8549698c2f648e57e1ca0b/users/ae2b410ff4b645beb42fde319048c68a/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:16:39.157420 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201beae-92cc-4c8f-b71c-e0aa7a4c50be None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:39.157885 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201beae-92cc-4c8f-b71c-e0aa7a4c50be None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:39.158238 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201beae-92cc-4c8f-b71c-e0aa7a4c50be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:39.205611 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-5201beae-92cc-4c8f-b71c-e0aa7a4c50be None None] MFA Rules not processed for user `ae2b410ff4b645beb42fde319048c68a`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:16:39.206933 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-5201beae-92cc-4c8f-b71c-e0aa7a4c50be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:39.270952 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4194ba6e-ac7e-410f-8a60-93bbb8462e27 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:39.273016 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4194ba6e-ac7e-410f-8a60-93bbb8462e27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:39.302157 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 949/1903] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:16:39 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:16:39.310980 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5201beae-92cc-4c8f-b71c-e0aa7a4c50be None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:39.312979 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-5201beae-92cc-4c8f-b71c-e0aa7a4c50be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:39.360328 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4194ba6e-ac7e-410f-8a60-93bbb8462e27 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:39.368574 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378212dd-adac-460a-90e3-02a8682c8495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:39.369627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378212dd-adac-460a-90e3-02a8682c8495 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:39.370010 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378212dd-adac-460a-90e3-02a8682c8495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:39.371169 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378212dd-adac-460a-90e3-02a8682c8495 admin admin] PATH_INFO: `/v3/projects/91d9bdd19b054dc68ac2fd70021390cf` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:39.373332 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378212dd-adac-460a-90e3-02a8682c8495 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91d9bdd19b054dc68ac2fd70021390cf)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:39.375739 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378212dd-adac-460a-90e3-02a8682c8495 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:39.398297 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5201beae-92cc-4c8f-b71c-e0aa7a4c50be None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:39.405369 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b9f67e-5204-4a6e-b003-cc68c103e6fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:39.406218 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9f67e-5204-4a6e-b003-cc68c103e6fc service cinder] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:39.406705 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9f67e-5204-4a6e-b003-cc68c103e6fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:39.406957 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9f67e-5204-4a6e-b003-cc68c103e6fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:39.424951 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b9f67e-5204-4a6e-b003-cc68c103e6fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:39.424951 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b9f67e-5204-4a6e-b003-cc68c103e6fc service cinder] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:39.445766 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-378212dd-adac-460a-90e3-02a8682c8495 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91d9bdd19b054dc68ac2fd70021390cf', 'request_id': 'req-378212dd-adac-460a-90e3-02a8682c8495'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:39.445873 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 950/1904] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:16:39 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:16:39.447502 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-378212dd-adac-460a-90e3-02a8682c8495 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91d9bdd19b054dc68ac2fd70021390cf', 'request_id': 'req-378212dd-adac-460a-90e3-02a8682c8495'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:39.459643 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-378212dd-adac-460a-90e3-02a8682c8495 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91d9bdd19b054dc68ac2fd70021390cf', 'request_id': 'req-378212dd-adac-460a-90e3-02a8682c8495'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:39.479688 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-378212dd-adac-460a-90e3-02a8682c8495 admin admin] The token cache is being invalidate because project 91d9bdd19b054dc68ac2fd70021390cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:16:39.479880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-378212dd-adac-460a-90e3-02a8682c8495 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:39.481170 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-378212dd-adac-460a-90e3-02a8682c8495 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:39.482843 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 955/1905] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:16:39 2022] DELETE /identity/v3/projects/91d9bdd19b054dc68ac2fd70021390cf => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:16:39.486900 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b9f67e-5204-4a6e-b003-cc68c103e6fc service cinder] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:39.497802 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b9f67e-5204-4a6e-b003-cc68c103e6fc service cinder] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:39.549269 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:39.560981 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727db7f5-39c7-42ca-ad24-a1fff5d2bc0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:39.561900 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727db7f5-39c7-42ca-ad24-a1fff5d2bc0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:39.562181 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727db7f5-39c7-42ca-ad24-a1fff5d2bc0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:39.562466 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727db7f5-39c7-42ca-ad24-a1fff5d2bc0a admin admin] PATH_INFO: `/v3/users/92ca758add884846a1513b353a9f6572` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:39.564281 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727db7f5-39c7-42ca-ad24-a1fff5d2bc0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=92ca758add884846a1513b353a9f6572)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:39.567974 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727db7f5-39c7-42ca-ad24-a1fff5d2bc0a admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:39.623289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-727db7f5-39c7-42ca-ad24-a1fff5d2bc0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92ca758add884846a1513b353a9f6572'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:39.623981 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-727db7f5-39c7-42ca-ad24-a1fff5d2bc0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92ca758add884846a1513b353a9f6572'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:39.637059 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-727db7f5-39c7-42ca-ad24-a1fff5d2bc0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92ca758add884846a1513b353a9f6572'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:39.644230 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-727db7f5-39c7-42ca-ad24-a1fff5d2bc0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92ca758add884846a1513b353a9f6572'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:39.644967 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-727db7f5-39c7-42ca-ad24-a1fff5d2bc0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92ca758add884846a1513b353a9f6572'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:39.663094 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 951/1906] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:16:39 2022] DELETE /identity/v3/users/92ca758add884846a1513b353a9f6572 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:16:39.991731 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378212dd-adac-460a-90e3-02a8682c8495 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:39.993213 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-378212dd-adac-460a-90e3-02a8682c8495 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:40.012949 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378212dd-adac-460a-90e3-02a8682c8495 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:40.089250 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d6f8c-3ea7-4fde-9eff-8f668b95d45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:40.090131 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d6f8c-3ea7-4fde-9eff-8f668b95d45b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:40.090948 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d6f8c-3ea7-4fde-9eff-8f668b95d45b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:40.091231 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d6f8c-3ea7-4fde-9eff-8f668b95d45b admin admin] PATH_INFO: `/v3/projects/7ed9eaf07b7149ed96f7e8df74034602` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:40.092513 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040d6f8c-3ea7-4fde-9eff-8f668b95d45b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ed9eaf07b7149ed96f7e8df74034602)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:40.097188 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040d6f8c-3ea7-4fde-9eff-8f668b95d45b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:40.189852 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727db7f5-39c7-42ca-ad24-a1fff5d2bc0a admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:40.191623 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-727db7f5-39c7-42ca-ad24-a1fff5d2bc0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:40.193117 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:40.223424 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727db7f5-39c7-42ca-ad24-a1fff5d2bc0a admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:40.225393 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-040d6f8c-3ea7-4fde-9eff-8f668b95d45b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ed9eaf07b7149ed96f7e8df74034602', 'request_id': 'req-040d6f8c-3ea7-4fde-9eff-8f668b95d45b'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:40.238785 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-040d6f8c-3ea7-4fde-9eff-8f668b95d45b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ed9eaf07b7149ed96f7e8df74034602', 'request_id': 'req-040d6f8c-3ea7-4fde-9eff-8f668b95d45b'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:40.257057 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-040d6f8c-3ea7-4fde-9eff-8f668b95d45b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ed9eaf07b7149ed96f7e8df74034602', 'request_id': 'req-040d6f8c-3ea7-4fde-9eff-8f668b95d45b'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:40.274719 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-040d6f8c-3ea7-4fde-9eff-8f668b95d45b admin admin] The token cache is being invalidate because project 7ed9eaf07b7149ed96f7e8df74034602 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:16:40.275153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-040d6f8c-3ea7-4fde-9eff-8f668b95d45b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:40.276130 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-040d6f8c-3ea7-4fde-9eff-8f668b95d45b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:40.284530 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 956/1907] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:16:39 2022] DELETE /identity/v3/projects/7ed9eaf07b7149ed96f7e8df74034602 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:16:40.286565 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b9fb7-0f3c-4a9b-a857-cfcff057d58c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:40.288399 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b9fb7-0f3c-4a9b-a857-cfcff057d58c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:40.288794 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b9fb7-0f3c-4a9b-a857-cfcff057d58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:40.329286 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-594b9fb7-0f3c-4a9b-a857-cfcff057d58c None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:16:40.333055 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-594b9fb7-0f3c-4a9b-a857-cfcff057d58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:40.373528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7097d-f60b-42f8-a4dc-f1be6fe5003a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:40.374809 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7097d-f60b-42f8-a4dc-f1be6fe5003a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:40.375301 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7097d-f60b-42f8-a4dc-f1be6fe5003a admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:40.375828 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7097d-f60b-42f8-a4dc-f1be6fe5003a admin admin] PATH_INFO: `/v3/users/8630010672024c7eac5c38f1828e74cb` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:40.378004 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b7097d-f60b-42f8-a4dc-f1be6fe5003a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8630010672024c7eac5c38f1828e74cb)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:40.380665 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b7097d-f60b-42f8-a4dc-f1be6fe5003a admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:40.427791 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 957/1908] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:16:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:16:40.444104 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32f040-ce81-480d-ae1e-025d5f0b3733 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:40.444457 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32f040-ce81-480d-ae1e-025d5f0b3733 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:40.444665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32f040-ce81-480d-ae1e-025d5f0b3733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:40.481387 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-12b7097d-f60b-42f8-a4dc-f1be6fe5003a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8630010672024c7eac5c38f1828e74cb'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:40.482453 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-12b7097d-f60b-42f8-a4dc-f1be6fe5003a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8630010672024c7eac5c38f1828e74cb'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:40.492282 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-ad32f040-ce81-480d-ae1e-025d5f0b3733 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:16:40.493589 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ad32f040-ce81-480d-ae1e-025d5f0b3733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:40.522115 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-12b7097d-f60b-42f8-a4dc-f1be6fe5003a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8630010672024c7eac5c38f1828e74cb'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:40.523729 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 958/1909] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:16:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:16:40.529406 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad32f040-ce81-480d-ae1e-025d5f0b3733 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:40.535899 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-12b7097d-f60b-42f8-a4dc-f1be6fe5003a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8630010672024c7eac5c38f1828e74cb'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:40.537414 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-12b7097d-f60b-42f8-a4dc-f1be6fe5003a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8630010672024c7eac5c38f1828e74cb'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:40.539019 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:40.546704 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad32f040-ce81-480d-ae1e-025d5f0b3733 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:40.565430 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 952/1910] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:16:40 2022] DELETE /identity/v3/users/8630010672024c7eac5c38f1828e74cb => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:16:40.567346 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6c89d1-d1a9-45e5-bdd0-39256ebb1f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:40.568057 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c89d1-d1a9-45e5-bdd0-39256ebb1f5f admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:40.568337 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c89d1-d1a9-45e5-bdd0-39256ebb1f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:40.569041 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da37afa-6b79-4283-ad94-9e1fc3b3e692 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:40.569359 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da37afa-6b79-4283-ad94-9e1fc3b3e692 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:40.569600 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da37afa-6b79-4283-ad94-9e1fc3b3e692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:40.570134 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c89d1-d1a9-45e5-bdd0-39256ebb1f5f admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:40.571631 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6c89d1-d1a9-45e5-bdd0-39256ebb1f5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:16:40.571631 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6c89d1-d1a9-45e5-bdd0-39256ebb1f5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:40.573022 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6c89d1-d1a9-45e5-bdd0-39256ebb1f5f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:40.594484 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 959/1911] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:16:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:16:40.595261 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-8da37afa-6b79-4283-ad94-9e1fc3b3e692 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:16:40.595996 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6c89d1-d1a9-45e5-bdd0-39256ebb1f5f admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:40.596570 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-8da37afa-6b79-4283-ad94-9e1fc3b3e692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:40.610743 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6c89d1-d1a9-45e5-bdd0-39256ebb1f5f admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:40.691485 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb954b7-4d4a-48e2-97fd-66ca5e2df364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:40.691747 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 953/1912] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:16:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:16:40.692206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb954b7-4d4a-48e2-97fd-66ca5e2df364 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:40.692489 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb954b7-4d4a-48e2-97fd-66ca5e2df364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:40.692865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb954b7-4d4a-48e2-97fd-66ca5e2df364 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:40.696490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb954b7-4d4a-48e2-97fd-66ca5e2df364 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:40.699583 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb954b7-4d4a-48e2-97fd-66ca5e2df364 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:40.705788 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da37afa-6b79-4283-ad94-9e1fc3b3e692 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:40.707993 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-8da37afa-6b79-4283-ad94-9e1fc3b3e692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:40.737009 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 960/1913] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:16:40 2022] POST /identity/v3/projects => generated 361 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:16:40.743075 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb954b7-4d4a-48e2-97fd-66ca5e2df364 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:40.796345 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:40.815886 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da37afa-6b79-4283-ad94-9e1fc3b3e692 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:40.844646 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb954b7-4d4a-48e2-97fd-66ca5e2df364 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:40.849744 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0893c33f-9f26-4395-bf0a-2b0f579ee4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:40.850819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0893c33f-9f26-4395-bf0a-2b0f579ee4e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:40.850819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0893c33f-9f26-4395-bf0a-2b0f579ee4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:40.851030 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0893c33f-9f26-4395-bf0a-2b0f579ee4e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:40.851661 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0893c33f-9f26-4395-bf0a-2b0f579ee4e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:40.853725 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0893c33f-9f26-4395-bf0a-2b0f579ee4e8 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:40.877326 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 961/1914] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:16:40 2022] POST /identity/v3/users => generated 281 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:16:40.882027 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0893c33f-9f26-4395-bf0a-2b0f579ee4e8 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:40.893744 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0893c33f-9f26-4395-bf0a-2b0f579ee4e8 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:40.895412 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a35c4-47b0-4496-92bf-4f5f56eea958 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:40.896518 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a35c4-47b0-4496-92bf-4f5f56eea958 service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:40.897059 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a35c4-47b0-4496-92bf-4f5f56eea958 service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:40.897490 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a35c4-47b0-4496-92bf-4f5f56eea958 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:40.899889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b25bd-f72b-4a22-8cab-fe103ec481e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:40.900563 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b25bd-f72b-4a22-8cab-fe103ec481e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:40.900845 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b25bd-f72b-4a22-8cab-fe103ec481e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:40.901119 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b25bd-f72b-4a22-8cab-fe103ec481e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:40.902447 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b25bd-f72b-4a22-8cab-fe103ec481e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:40.904174 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b25bd-f72b-4a22-8cab-fe103ec481e5 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:40.911632 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4a35c4-47b0-4496-92bf-4f5f56eea958 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:40.913150 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4a35c4-47b0-4496-92bf-4f5f56eea958 service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:40.915438 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 962/1915] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:16:40.919317 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b25bd-f72b-4a22-8cab-fe103ec481e5 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:40.924682 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 954/1916] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:16:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:16:40.929498 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b25bd-f72b-4a22-8cab-fe103ec481e5 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:40.935000 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe16299-1959-42b4-9789-dd1f8dd3fd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:40.935992 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe16299-1959-42b4-9789-dd1f8dd3fd36 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:40.936379 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe16299-1959-42b4-9789-dd1f8dd3fd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:40.936767 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe16299-1959-42b4-9789-dd1f8dd3fd36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:40.938156 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe16299-1959-42b4-9789-dd1f8dd3fd36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:40.939892 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe16299-1959-42b4-9789-dd1f8dd3fd36 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:40.955385 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 963/1917] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:16:40.956674 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a35c4-47b0-4496-92bf-4f5f56eea958 service neutron] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:40.967571 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a35c4-47b0-4496-92bf-4f5f56eea958 service neutron] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:40.973307 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d11543-1a8c-43ff-a000-4c95b910d0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:40.974145 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d11543-1a8c-43ff-a000-4c95b910d0d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:40.974145 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d11543-1a8c-43ff-a000-4c95b910d0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:40.974367 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d11543-1a8c-43ff-a000-4c95b910d0d7 admin admin] PATH_INFO: `/v3/projects/4a1b4b84023643888ea95c354ce33b23/users/ecfbe0eb67564df7a85029a345abc546/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:40.993317 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d11543-1a8c-43ff-a000-4c95b910d0d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a1b4b84023643888ea95c354ce33b23, user_id=ecfbe0eb67564df7a85029a345abc546, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:40.997270 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d11543-1a8c-43ff-a000-4c95b910d0d7 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:41.017179 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 955/1918] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:16:40 2022] PUT /identity/v3/projects/4a1b4b84023643888ea95c354ce33b23/users/ecfbe0eb67564df7a85029a345abc546/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:16:41.029784 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b961aa-378e-481f-85c6-22f71ce01737 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:41.030336 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b961aa-378e-481f-85c6-22f71ce01737 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:41.030336 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b961aa-378e-481f-85c6-22f71ce01737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:41.069453 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-76b961aa-378e-481f-85c6-22f71ce01737 None None] MFA Rules not processed for user `ecfbe0eb67564df7a85029a345abc546`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:16:41.071142 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-76b961aa-378e-481f-85c6-22f71ce01737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:41.199582 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 964/1919] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:16:41 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:16:41.203561 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d11543-1a8c-43ff-a000-4c95b910d0d7 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:41.247820 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:INSERT=2,keystone:DELETE=1,keystone:SELECT=97 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:41.289093 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d11543-1a8c-43ff-a000-4c95b910d0d7 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:41.295921 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a04470-60e4-4d68-98cd-77d53a37d83d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:41.296705 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a04470-60e4-4d68-98cd-77d53a37d83d admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:41.296978 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a04470-60e4-4d68-98cd-77d53a37d83d admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:41.297393 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a04470-60e4-4d68-98cd-77d53a37d83d admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:41.298202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a04470-60e4-4d68-98cd-77d53a37d83d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:41.301218 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a04470-60e4-4d68-98cd-77d53a37d83d admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:41.336766 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 956/1920] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:16:41 2022] POST /identity/v3/projects => generated 365 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:16:41.338345 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b961aa-378e-481f-85c6-22f71ce01737 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:41.339995 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-76b961aa-378e-481f-85c6-22f71ce01737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:41.345190 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a04470-60e4-4d68-98cd-77d53a37d83d admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:41.361709 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b961aa-378e-481f-85c6-22f71ce01737 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:41.436113 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a04470-60e4-4d68-98cd-77d53a37d83d admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:41.441999 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee7c2c5-f24d-4db5-a67a-afb548acbada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:41.442786 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7c2c5-f24d-4db5-a67a-afb548acbada admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:41.443169 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7c2c5-f24d-4db5-a67a-afb548acbada admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:41.443452 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7c2c5-f24d-4db5-a67a-afb548acbada admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:41.444276 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee7c2c5-f24d-4db5-a67a-afb548acbada admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:41.446558 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5793c86-87b5-4abb-9baa-2670b897a1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:41.447551 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee7c2c5-f24d-4db5-a67a-afb548acbada admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:41.447656 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5793c86-87b5-4abb-9baa-2670b897a1b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:41.447927 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5793c86-87b5-4abb-9baa-2670b897a1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:41.448913 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5793c86-87b5-4abb-9baa-2670b897a1b8 admin admin] PATH_INFO: `/v3/projects/96d5e99d8e394573b45abbb022e4d166` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:41.452331 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5793c86-87b5-4abb-9baa-2670b897a1b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96d5e99d8e394573b45abbb022e4d166)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:41.454272 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5793c86-87b5-4abb-9baa-2670b897a1b8 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:41.471145 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 957/1921] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:16:41 2022] POST /identity/v3/users => generated 289 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:16:41.475439 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee7c2c5-f24d-4db5-a67a-afb548acbada admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:41.488826 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee7c2c5-f24d-4db5-a67a-afb548acbada admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:41.497061 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c50add-5999-4b1d-98ae-14654dfeba78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:41.498023 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c50add-5999-4b1d-98ae-14654dfeba78 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:41.498347 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c50add-5999-4b1d-98ae-14654dfeba78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:41.498725 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c50add-5999-4b1d-98ae-14654dfeba78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:41.499521 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:41.499928 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c50add-5999-4b1d-98ae-14654dfeba78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:41.501922 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c50add-5999-4b1d-98ae-14654dfeba78 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:41.518195 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 958/1922] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:16:41 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:16:41.522258 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c50add-5999-4b1d-98ae-14654dfeba78 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:41.524194 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e5793c86-87b5-4abb-9baa-2670b897a1b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96d5e99d8e394573b45abbb022e4d166', 'request_id': 'req-e5793c86-87b5-4abb-9baa-2670b897a1b8'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:41.532446 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e5793c86-87b5-4abb-9baa-2670b897a1b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96d5e99d8e394573b45abbb022e4d166', 'request_id': 'req-e5793c86-87b5-4abb-9baa-2670b897a1b8'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:41.536364 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c50add-5999-4b1d-98ae-14654dfeba78 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:41.544211 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e5793c86-87b5-4abb-9baa-2670b897a1b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96d5e99d8e394573b45abbb022e4d166', 'request_id': 'req-e5793c86-87b5-4abb-9baa-2670b897a1b8'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:41.557113 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e5793c86-87b5-4abb-9baa-2670b897a1b8 admin admin] The token cache is being invalidate because project 96d5e99d8e394573b45abbb022e4d166 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:16:41.558125 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e5793c86-87b5-4abb-9baa-2670b897a1b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:41.558886 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e5793c86-87b5-4abb-9baa-2670b897a1b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:41.560661 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 965/1923] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:16:41 2022] DELETE /identity/v3/projects/96d5e99d8e394573b45abbb022e4d166 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:16:41.564662 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5793c86-87b5-4abb-9baa-2670b897a1b8 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:41.566167 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e5793c86-87b5-4abb-9baa-2670b897a1b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:41.627868 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d388788-a62d-456e-aeaf-73f473068045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:41.632494 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d388788-a62d-456e-aeaf-73f473068045 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:41.633077 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d388788-a62d-456e-aeaf-73f473068045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:41.633775 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d388788-a62d-456e-aeaf-73f473068045 admin admin] PATH_INFO: `/v3/projects/9ddb265f3807450991a434ef4210453b/users/b5230991c9164a5db60943652643455a/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:41.659667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d388788-a62d-456e-aeaf-73f473068045 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ddb265f3807450991a434ef4210453b, user_id=b5230991c9164a5db60943652643455a, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:41.665410 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d388788-a62d-456e-aeaf-73f473068045 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:41.669816 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5793c86-87b5-4abb-9baa-2670b897a1b8 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:41.676984 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52878d82-c163-4ebb-aadd-e42d5d004cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:41.678642 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52878d82-c163-4ebb-aadd-e42d5d004cdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:41.679007 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52878d82-c163-4ebb-aadd-e42d5d004cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:41.679375 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52878d82-c163-4ebb-aadd-e42d5d004cdf admin admin] PATH_INFO: `/v3/users/ae2b410ff4b645beb42fde319048c68a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:41.681621 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52878d82-c163-4ebb-aadd-e42d5d004cdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae2b410ff4b645beb42fde319048c68a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:41.684329 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52878d82-c163-4ebb-aadd-e42d5d004cdf admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:41.686792 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 959/1924] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:16:41 2022] PUT /identity/v3/projects/9ddb265f3807450991a434ef4210453b/users/b5230991c9164a5db60943652643455a/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:16:41.690836 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d388788-a62d-456e-aeaf-73f473068045 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:41.704444 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-5d388788-a62d-456e-aeaf-73f473068045 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:41.721558 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:41.743329 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-52878d82-c163-4ebb-aadd-e42d5d004cdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae2b410ff4b645beb42fde319048c68a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:41.744371 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-52878d82-c163-4ebb-aadd-e42d5d004cdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae2b410ff4b645beb42fde319048c68a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:41.761030 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-52878d82-c163-4ebb-aadd-e42d5d004cdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae2b410ff4b645beb42fde319048c68a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:41.768614 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-52878d82-c163-4ebb-aadd-e42d5d004cdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae2b410ff4b645beb42fde319048c68a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:41.778391 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-52878d82-c163-4ebb-aadd-e42d5d004cdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae2b410ff4b645beb42fde319048c68a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:41.782285 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d388788-a62d-456e-aeaf-73f473068045 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:41.789270 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 966/1925] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:16:41 2022] DELETE /identity/v3/users/ae2b410ff4b645beb42fde319048c68a => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:16:41.789412 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74671b4-1c31-48a5-aa81-aac381220fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:41.789412 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74671b4-1c31-48a5-aa81-aac381220fa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:41.789412 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74671b4-1c31-48a5-aa81-aac381220fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:41.789412 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74671b4-1c31-48a5-aa81-aac381220fa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:41.790331 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74671b4-1c31-48a5-aa81-aac381220fa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:41.792440 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74671b4-1c31-48a5-aa81-aac381220fa2 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:41.804892 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 960/1926] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:16:41 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:16:41.809703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52878d82-c163-4ebb-aadd-e42d5d004cdf admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:41.812122 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-52878d82-c163-4ebb-aadd-e42d5d004cdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:41.858217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:41.903062 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52878d82-c163-4ebb-aadd-e42d5d004cdf admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:41.909062 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b0d2f8-bc7a-4f12-9370-16c6c3e5fda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:41.910054 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0d2f8-bc7a-4f12-9370-16c6c3e5fda8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:41.910366 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0d2f8-bc7a-4f12-9370-16c6c3e5fda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:41.910645 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0d2f8-bc7a-4f12-9370-16c6c3e5fda8 admin admin] PATH_INFO: `/v3/projects/9ddb265f3807450991a434ef4210453b/users/b5230991c9164a5db60943652643455a/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:41.912714 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b0d2f8-bc7a-4f12-9370-16c6c3e5fda8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ddb265f3807450991a434ef4210453b, user_id=b5230991c9164a5db60943652643455a, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:41.915421 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b0d2f8-bc7a-4f12-9370-16c6c3e5fda8 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:41.927927 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 967/1927] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:16:41 2022] PUT /identity/v3/projects/9ddb265f3807450991a434ef4210453b/users/b5230991c9164a5db60943652643455a/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:16:41.940884 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90119d0a-d2e9-4c65-b738-7b1311762525 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:41.940884 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90119d0a-d2e9-4c65-b738-7b1311762525 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:41.941177 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90119d0a-d2e9-4c65-b738-7b1311762525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:41.973549 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-90119d0a-d2e9-4c65-b738-7b1311762525 None None] MFA Rules not processed for user `b5230991c9164a5db60943652643455a`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:16:41.975094 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-90119d0a-d2e9-4c65-b738-7b1311762525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:42.087358 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 961/1928] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:16:41 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:16:42.095039 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b0d2f8-bc7a-4f12-9370-16c6c3e5fda8 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:42.098827 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-60b0d2f8-bc7a-4f12-9370-16c6c3e5fda8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:42.186223 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b0d2f8-bc7a-4f12-9370-16c6c3e5fda8 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:42.191116 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1419fbc-b9eb-4f2f-9268-5c450df04d76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:42.191842 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1419fbc-b9eb-4f2f-9268-5c450df04d76 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:42.192131 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1419fbc-b9eb-4f2f-9268-5c450df04d76 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:42.192412 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1419fbc-b9eb-4f2f-9268-5c450df04d76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:42.211829 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1419fbc-b9eb-4f2f-9268-5c450df04d76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:42.215426 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1419fbc-b9eb-4f2f-9268-5c450df04d76 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:42.232857 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 968/1929] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:16:42 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:16:42.320869 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90119d0a-d2e9-4c65-b738-7b1311762525 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:42.332449 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90119d0a-d2e9-4c65-b738-7b1311762525 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:42.339202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32dccf0-2096-4418-bcb7-38f8794df0ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:42.339962 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32dccf0-2096-4418-bcb7-38f8794df0ed service cinder] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:42.340245 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32dccf0-2096-4418-bcb7-38f8794df0ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:42.340521 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32dccf0-2096-4418-bcb7-38f8794df0ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:42.343772 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-f32dccf0-2096-4418-bcb7-38f8794df0ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:42.380111 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1419fbc-b9eb-4f2f-9268-5c450df04d76 service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:42.383389 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a1419fbc-b9eb-4f2f-9268-5c450df04d76 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:42.441284 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32dccf0-2096-4418-bcb7-38f8794df0ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:42.444371 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32dccf0-2096-4418-bcb7-38f8794df0ed service cinder] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:42.458902 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 962/1930] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:16:42 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:16:42.511082 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1419fbc-b9eb-4f2f-9268-5c450df04d76 service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:42.517340 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2fa5ff-a954-433c-919d-e2f203542f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:42.518367 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2fa5ff-a954-433c-919d-e2f203542f10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:42.518654 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2fa5ff-a954-433c-919d-e2f203542f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:42.518961 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2fa5ff-a954-433c-919d-e2f203542f10 admin admin] PATH_INFO: `/v3/projects/41eacb06bb8549698c2f648e57e1ca0b` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:42.520209 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2fa5ff-a954-433c-919d-e2f203542f10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41eacb06bb8549698c2f648e57e1ca0b)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:42.522140 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2fa5ff-a954-433c-919d-e2f203542f10 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:42.598963 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-da2fa5ff-a954-433c-919d-e2f203542f10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41eacb06bb8549698c2f648e57e1ca0b', 'request_id': 'req-da2fa5ff-a954-433c-919d-e2f203542f10'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:42.607800 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-da2fa5ff-a954-433c-919d-e2f203542f10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41eacb06bb8549698c2f648e57e1ca0b', 'request_id': 'req-da2fa5ff-a954-433c-919d-e2f203542f10'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:42.619378 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-da2fa5ff-a954-433c-919d-e2f203542f10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41eacb06bb8549698c2f648e57e1ca0b', 'request_id': 'req-da2fa5ff-a954-433c-919d-e2f203542f10'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:42.629335 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:42.634159 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-da2fa5ff-a954-433c-919d-e2f203542f10 admin admin] The token cache is being invalidate because project 41eacb06bb8549698c2f648e57e1ca0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:16:42.634657 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-da2fa5ff-a954-433c-919d-e2f203542f10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:42.635422 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-da2fa5ff-a954-433c-919d-e2f203542f10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:42.638696 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 969/1931] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:16:42 2022] DELETE /identity/v3/projects/41eacb06bb8549698c2f648e57e1ca0b => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:16:42.646698 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d098a-0880-4409-96f4-306272476384 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:42.648629 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d098a-0880-4409-96f4-306272476384 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:42.649040 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d098a-0880-4409-96f4-306272476384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:42.724245 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-930d098a-0880-4409-96f4-306272476384 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:16:42.725620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-930d098a-0880-4409-96f4-306272476384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:42.777924 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:42.824729 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 963/1932] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:16:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:16:42.832093 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d98c3a-da59-4d17-b3ef-73d487d47d5d None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:42.832465 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d98c3a-da59-4d17-b3ef-73d487d47d5d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:42.832821 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d98c3a-da59-4d17-b3ef-73d487d47d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:42.853523 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-d8d98c3a-da59-4d17-b3ef-73d487d47d5d None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:16:42.855170 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d8d98c3a-da59-4d17-b3ef-73d487d47d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:42.865496 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 970/1933] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:16:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:16:42.869387 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d098a-0880-4409-96f4-306272476384 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:42.881765 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d098a-0880-4409-96f4-306272476384 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:42.886967 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c678d6e5-d4b8-4b0d-8575-84b9210f257c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:42.887694 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678d6e5-d4b8-4b0d-8575-84b9210f257c admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:42.887987 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678d6e5-d4b8-4b0d-8575-84b9210f257c admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:42.888249 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678d6e5-d4b8-4b0d-8575-84b9210f257c admin admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:42.889016 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c678d6e5-d4b8-4b0d-8575-84b9210f257c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:16:42.889354 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c678d6e5-d4b8-4b0d-8575-84b9210f257c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:42.890992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c678d6e5-d4b8-4b0d-8575-84b9210f257c admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:42.905799 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 964/1934] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:16:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:16:42.910321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d98c3a-da59-4d17-b3ef-73d487d47d5d None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:42.921176 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d98c3a-da59-4d17-b3ef-73d487d47d5d None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:42.927434 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f2858-d22b-4239-8251-8de524d1f083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:42.928426 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f2858-d22b-4239-8251-8de524d1f083 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:42.929081 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f2858-d22b-4239-8251-8de524d1f083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:42.929376 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f2858-d22b-4239-8251-8de524d1f083 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:42.930615 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f2858-d22b-4239-8251-8de524d1f083 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:42.932643 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f2858-d22b-4239-8251-8de524d1f083 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:42.967660 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 971/1935] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:16:42 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:16:42.969351 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c678d6e5-d4b8-4b0d-8575-84b9210f257c admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:42.980518 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c678d6e5-d4b8-4b0d-8575-84b9210f257c admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:43.041440 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff579b42-b019-4b7f-866a-49c3854262f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:43.042736 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff579b42-b019-4b7f-866a-49c3854262f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:43.043018 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff579b42-b019-4b7f-866a-49c3854262f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:43.043295 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff579b42-b019-4b7f-866a-49c3854262f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:43.044064 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff579b42-b019-4b7f-866a-49c3854262f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:43.046405 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff579b42-b019-4b7f-866a-49c3854262f5 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:43.068936 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 965/1936] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:16:42 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:16:43.072717 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f2858-d22b-4239-8251-8de524d1f083 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:43.083231 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f2858-d22b-4239-8251-8de524d1f083 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:43.088362 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc36b45-4b0f-4c5e-8b6d-1f47b7084bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:43.089630 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc36b45-4b0f-4c5e-8b6d-1f47b7084bb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:43.089974 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc36b45-4b0f-4c5e-8b6d-1f47b7084bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:43.090338 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc36b45-4b0f-4c5e-8b6d-1f47b7084bb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:43.091113 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc36b45-4b0f-4c5e-8b6d-1f47b7084bb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:43.092800 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc36b45-4b0f-4c5e-8b6d-1f47b7084bb3 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:43.105083 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 972/1937] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:16:43.109197 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff579b42-b019-4b7f-866a-49c3854262f5 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:43.119555 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff579b42-b019-4b7f-866a-49c3854262f5 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:43.124570 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16398158-4dd5-4278-b158-9bc445d8e7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:43.125253 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16398158-4dd5-4278-b158-9bc445d8e7a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:43.125528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16398158-4dd5-4278-b158-9bc445d8e7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:43.125803 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16398158-4dd5-4278-b158-9bc445d8e7a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:43.126967 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16398158-4dd5-4278-b158-9bc445d8e7a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:43.128777 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16398158-4dd5-4278-b158-9bc445d8e7a3 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:43.144020 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 966/1938] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:16:43.148185 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc36b45-4b0f-4c5e-8b6d-1f47b7084bb3 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:43.160354 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc36b45-4b0f-4c5e-8b6d-1f47b7084bb3 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:43.165414 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b1ac2-c8b5-4c9f-ba4d-cf9aa6cf762a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:43.166315 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b1ac2-c8b5-4c9f-ba4d-cf9aa6cf762a admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:43.166593 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b1ac2-c8b5-4c9f-ba4d-cf9aa6cf762a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:43.166867 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b1ac2-c8b5-4c9f-ba4d-cf9aa6cf762a admin admin] PATH_INFO: `/v3/projects/6cc4805d9511488ca2759f2aef15a357/users/3a8e5e7596a44fac93dd07e9d17e7566/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:43.184215 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4b1ac2-c8b5-4c9f-ba4d-cf9aa6cf762a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cc4805d9511488ca2759f2aef15a357, user_id=3a8e5e7596a44fac93dd07e9d17e7566, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:43.187070 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4b1ac2-c8b5-4c9f-ba4d-cf9aa6cf762a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:43.204621 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 973/1939] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:16:43 2022] PUT /identity/v3/projects/6cc4805d9511488ca2759f2aef15a357/users/3a8e5e7596a44fac93dd07e9d17e7566/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:16:43.209806 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef7d6d-7f9c-4224-ab10-f741fd3989bb None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:43.210099 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef7d6d-7f9c-4224-ab10-f741fd3989bb None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:43.210380 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef7d6d-7f9c-4224-ab10-f741fd3989bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:43.230309 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-ccef7d6d-7f9c-4224-ab10-f741fd3989bb None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:16:43.231669 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-ccef7d6d-7f9c-4224-ab10-f741fd3989bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:43.306215 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 967/1940] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:16:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:16:43.339309 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b1ac2-c8b5-4c9f-ba4d-cf9aa6cf762a admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:43.341560 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3f4b1ac2-c8b5-4c9f-ba4d-cf9aa6cf762a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:43.435649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b1ac2-c8b5-4c9f-ba4d-cf9aa6cf762a admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:43.441826 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9a7e6-d73e-4a45-86ed-47146578e9cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:43.442502 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9a7e6-d73e-4a45-86ed-47146578e9cd service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:43.442877 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9a7e6-d73e-4a45-86ed-47146578e9cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:43.443186 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9a7e6-d73e-4a45-86ed-47146578e9cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:43.455168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b9a7e6-d73e-4a45-86ed-47146578e9cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:43.456716 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b9a7e6-d73e-4a45-86ed-47146578e9cd service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:43.467907 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 974/1941] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:16:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:16:44.432683 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2bc69-ead9-4de1-a0b5-f1f5e44c6dc0 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:44.433212 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2bc69-ead9-4de1-a0b5-f1f5e44c6dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:44.433212 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2bc69-ead9-4de1-a0b5-f1f5e44c6dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:44.517356 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-d8c2bc69-ead9-4de1-a0b5-f1f5e44c6dc0 None None] MFA Rules not processed for user `3a8e5e7596a44fac93dd07e9d17e7566`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:16:44.518701 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-d8c2bc69-ead9-4de1-a0b5-f1f5e44c6dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:44.595184 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:44.657696 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 968/1942] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:16:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:16:44.658531 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9a7e6-d73e-4a45-86ed-47146578e9cd service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:44.669427 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9a7e6-d73e-4a45-86ed-47146578e9cd service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:44.674906 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eeb6b8-2469-49d3-8c2e-8ac92e9e69d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:44.675691 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eeb6b8-2469-49d3-8c2e-8ac92e9e69d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:44.675980 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eeb6b8-2469-49d3-8c2e-8ac92e9e69d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:44.676259 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eeb6b8-2469-49d3-8c2e-8ac92e9e69d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:44.677620 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eeb6b8-2469-49d3-8c2e-8ac92e9e69d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:44.679884 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eeb6b8-2469-49d3-8c2e-8ac92e9e69d5 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:44.712759 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 975/1943] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:16:44 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:16:44.717953 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c2bc69-ead9-4de1-a0b5-f1f5e44c6dc0 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:44.732712 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c2bc69-ead9-4de1-a0b5-f1f5e44c6dc0 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:44.802869 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c26ccf7-ed35-4a6f-811f-52b46b88f205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:44.803743 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c26ccf7-ed35-4a6f-811f-52b46b88f205 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:44.804096 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c26ccf7-ed35-4a6f-811f-52b46b88f205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:44.804737 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c26ccf7-ed35-4a6f-811f-52b46b88f205 admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:44.805563 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c26ccf7-ed35-4a6f-811f-52b46b88f205 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:44.807703 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c26ccf7-ed35-4a6f-811f-52b46b88f205 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:44.834512 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 969/1944] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:16:44 2022] POST /identity/v3/users => generated 292 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:16:44.838263 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eeb6b8-2469-49d3-8c2e-8ac92e9e69d5 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:44.854024 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eeb6b8-2469-49d3-8c2e-8ac92e9e69d5 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:44.863967 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af09a244-fc4f-4cbf-8260-ba09ac396ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:44.864673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af09a244-fc4f-4cbf-8260-ba09ac396ea0 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:44.865155 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af09a244-fc4f-4cbf-8260-ba09ac396ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:44.865431 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af09a244-fc4f-4cbf-8260-ba09ac396ea0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:44.866241 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af09a244-fc4f-4cbf-8260-ba09ac396ea0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:44.867941 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af09a244-fc4f-4cbf-8260-ba09ac396ea0 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:44.880914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 976/1945] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:16:44.886586 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c26ccf7-ed35-4a6f-811f-52b46b88f205 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:44.898293 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c26ccf7-ed35-4a6f-811f-52b46b88f205 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:44.904701 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab8d0b-d3e0-481c-9950-481d9811e275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:44.905970 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab8d0b-d3e0-481c-9950-481d9811e275 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:44.905970 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab8d0b-d3e0-481c-9950-481d9811e275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:44.906165 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ab8d0b-d3e0-481c-9950-481d9811e275 admin admin] PATH_INFO: `/v3/projects/632cfd2af352489ea38e08274d28274e/users/ffa766792e7448da8a64eba9e2f07aa8/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:44.925816 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ab8d0b-d3e0-481c-9950-481d9811e275 admin admin] RBAC: Authorizing `identity:create_grant(project_id=632cfd2af352489ea38e08274d28274e, user_id=ffa766792e7448da8a64eba9e2f07aa8, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:44.928839 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ab8d0b-d3e0-481c-9950-481d9811e275 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:44.946768 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 970/1946] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:16:44 2022] PUT /identity/v3/projects/632cfd2af352489ea38e08274d28274e/users/ffa766792e7448da8a64eba9e2f07aa8/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:16:44.951442 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af09a244-fc4f-4cbf-8260-ba09ac396ea0 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:44.964940 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af09a244-fc4f-4cbf-8260-ba09ac396ea0 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:45.031466 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2758f0fa-8de7-46c6-82dc-ab5ce1ffac1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:45.032179 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758f0fa-8de7-46c6-82dc-ab5ce1ffac1c admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:45.032461 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758f0fa-8de7-46c6-82dc-ab5ce1ffac1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:45.032740 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758f0fa-8de7-46c6-82dc-ab5ce1ffac1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:45.033472 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2758f0fa-8de7-46c6-82dc-ab5ce1ffac1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:45.035077 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2758f0fa-8de7-46c6-82dc-ab5ce1ffac1c admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:45.045410 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 977/1947] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:16:45.049827 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab8d0b-d3e0-481c-9950-481d9811e275 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:45.061432 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ab8d0b-d3e0-481c-9950-481d9811e275 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:45.066135 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b9dedf-ec7d-4d15-8b2d-953ab23b3237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:45.067020 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9dedf-ec7d-4d15-8b2d-953ab23b3237 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:45.067302 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9dedf-ec7d-4d15-8b2d-953ab23b3237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:45.067633 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b9dedf-ec7d-4d15-8b2d-953ab23b3237 admin admin] PATH_INFO: `/v3/projects/632cfd2af352489ea38e08274d28274e/users/ffa766792e7448da8a64eba9e2f07aa8/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:45.069613 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b9dedf-ec7d-4d15-8b2d-953ab23b3237 admin admin] RBAC: Authorizing `identity:create_grant(project_id=632cfd2af352489ea38e08274d28274e, user_id=ffa766792e7448da8a64eba9e2f07aa8, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:45.072291 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b9dedf-ec7d-4d15-8b2d-953ab23b3237 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:45.083792 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 971/1948] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:16:45 2022] PUT /identity/v3/projects/632cfd2af352489ea38e08274d28274e/users/ffa766792e7448da8a64eba9e2f07aa8/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:16:45.747054 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2758f0fa-8de7-46c6-82dc-ab5ce1ffac1c admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:45.756177 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-2758f0fa-8de7-46c6-82dc-ab5ce1ffac1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:45.765706 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:45.772262 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2758f0fa-8de7-46c6-82dc-ab5ce1ffac1c admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:45.781020 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda55456-2501-4125-8258-172e6bb92703 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:45.782316 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda55456-2501-4125-8258-172e6bb92703 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:45.782597 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda55456-2501-4125-8258-172e6bb92703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:45.819419 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-eda55456-2501-4125-8258-172e6bb92703 None None] MFA Rules not processed for user `ffa766792e7448da8a64eba9e2f07aa8`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:16:45.820737 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-eda55456-2501-4125-8258-172e6bb92703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:45.852575 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bed264-2ae3-4296-b6b6-e1f4cdb162cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:45.853467 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bed264-2ae3-4296-b6b6-e1f4cdb162cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:45.853755 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bed264-2ae3-4296-b6b6-e1f4cdb162cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:45.854197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bed264-2ae3-4296-b6b6-e1f4cdb162cb admin admin] PATH_INFO: `/v3/users/ecfbe0eb67564df7a85029a345abc546` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:45.855880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bed264-2ae3-4296-b6b6-e1f4cdb162cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecfbe0eb67564df7a85029a345abc546)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:45.857833 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bed264-2ae3-4296-b6b6-e1f4cdb162cb admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:45.905428 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 972/1949] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:16:45 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:16:45.917517 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda55456-2501-4125-8258-172e6bb92703 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:45.919202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-eda55456-2501-4125-8258-172e6bb92703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:45.927382 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-45bed264-2ae3-4296-b6b6-e1f4cdb162cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecfbe0eb67564df7a85029a345abc546'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:45.928496 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-45bed264-2ae3-4296-b6b6-e1f4cdb162cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecfbe0eb67564df7a85029a345abc546'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:45.930542 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:45.946747 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-45bed264-2ae3-4296-b6b6-e1f4cdb162cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecfbe0eb67564df7a85029a345abc546'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:45.956189 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-45bed264-2ae3-4296-b6b6-e1f4cdb162cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecfbe0eb67564df7a85029a345abc546'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:45.972630 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-45bed264-2ae3-4296-b6b6-e1f4cdb162cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecfbe0eb67564df7a85029a345abc546'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:45.984076 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 978/1950] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:16:45 2022] DELETE /identity/v3/users/ecfbe0eb67564df7a85029a345abc546 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:16:45.985485 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb53948-fb1a-4cd4-8a60-a1fb8ff6792d None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:45.985895 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb53948-fb1a-4cd4-8a60-a1fb8ff6792d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:45.986048 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb53948-fb1a-4cd4-8a60-a1fb8ff6792d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:45.994097 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda55456-2501-4125-8258-172e6bb92703 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:46.005364 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-8fb53948-fb1a-4cd4-8a60-a1fb8ff6792d None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:16:46.006596 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8fb53948-fb1a-4cd4-8a60-a1fb8ff6792d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:46.052387 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327d8521-0ed6-4b6d-aef7-5c550c982068 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:46.053203 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d8521-0ed6-4b6d-aef7-5c550c982068 service cinder] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:46.054098 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d8521-0ed6-4b6d-aef7-5c550c982068 service cinder] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:46.054748 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d8521-0ed6-4b6d-aef7-5c550c982068 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:46.057603 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-327d8521-0ed6-4b6d-aef7-5c550c982068 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:46.078977 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 979/1951] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:16:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:16:46.093122 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb53948-fb1a-4cd4-8a60-a1fb8ff6792d None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:46.094768 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8fb53948-fb1a-4cd4-8a60-a1fb8ff6792d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:46.123678 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327d8521-0ed6-4b6d-aef7-5c550c982068 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:46.124649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:46.126192 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327d8521-0ed6-4b6d-aef7-5c550c982068 service cinder] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:46.138130 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 973/1952] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:16:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:16:46.173086 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb53948-fb1a-4cd4-8a60-a1fb8ff6792d None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:46.177234 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb75b55-ddd0-4bf4-8246-c634dd3af95c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:46.177948 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb75b55-ddd0-4bf4-8246-c634dd3af95c service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:46.178296 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb75b55-ddd0-4bf4-8246-c634dd3af95c service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:46.178694 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb75b55-ddd0-4bf4-8246-c634dd3af95c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:46.193465 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb75b55-ddd0-4bf4-8246-c634dd3af95c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:46.194654 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb75b55-ddd0-4bf4-8246-c634dd3af95c service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:46.206135 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 980/1953] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:16:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:16:46.637066 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327d8521-0ed6-4b6d-aef7-5c550c982068 service cinder] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:46.639091 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-327d8521-0ed6-4b6d-aef7-5c550c982068 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:46.667463 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327d8521-0ed6-4b6d-aef7-5c550c982068 service cinder] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:46.673812 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2b62e0-888b-42ce-9b1e-931c805e0d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:46.675080 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b62e0-888b-42ce-9b1e-931c805e0d79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:46.675080 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b62e0-888b-42ce-9b1e-931c805e0d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:46.675374 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b62e0-888b-42ce-9b1e-931c805e0d79 admin admin] PATH_INFO: `/v3/projects/4a1b4b84023643888ea95c354ce33b23` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:46.676647 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2b62e0-888b-42ce-9b1e-931c805e0d79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a1b4b84023643888ea95c354ce33b23)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:46.680671 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2b62e0-888b-42ce-9b1e-931c805e0d79 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:46.740755 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-7c2b62e0-888b-42ce-9b1e-931c805e0d79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a1b4b84023643888ea95c354ce33b23', 'request_id': 'req-7c2b62e0-888b-42ce-9b1e-931c805e0d79'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:46.748355 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-7c2b62e0-888b-42ce-9b1e-931c805e0d79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a1b4b84023643888ea95c354ce33b23', 'request_id': 'req-7c2b62e0-888b-42ce-9b1e-931c805e0d79'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:46.756015 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-7c2b62e0-888b-42ce-9b1e-931c805e0d79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a1b4b84023643888ea95c354ce33b23', 'request_id': 'req-7c2b62e0-888b-42ce-9b1e-931c805e0d79'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:46.764140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:46.768648 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-7c2b62e0-888b-42ce-9b1e-931c805e0d79 admin admin] The token cache is being invalidate because project 4a1b4b84023643888ea95c354ce33b23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:16:46.769021 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-7c2b62e0-888b-42ce-9b1e-931c805e0d79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:46.770326 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-7c2b62e0-888b-42ce-9b1e-931c805e0d79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:46.772015 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 974/1954] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:16:46 2022] DELETE /identity/v3/projects/4a1b4b84023643888ea95c354ce33b23 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:16:46.776026 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb75b55-ddd0-4bf4-8246-c634dd3af95c service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:46.785666 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb75b55-ddd0-4bf4-8246-c634dd3af95c service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:46.842291 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76964af2-5008-4e88-ab15-3c4a04319d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:46.843098 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76964af2-5008-4e88-ab15-3c4a04319d0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:46.843380 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76964af2-5008-4e88-ab15-3c4a04319d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:46.843931 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76964af2-5008-4e88-ab15-3c4a04319d0b admin admin] PATH_INFO: `/v3/users/b5230991c9164a5db60943652643455a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:46.844975 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76964af2-5008-4e88-ab15-3c4a04319d0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5230991c9164a5db60943652643455a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:46.847404 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76964af2-5008-4e88-ab15-3c4a04319d0b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:46.895788 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-76964af2-5008-4e88-ab15-3c4a04319d0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5230991c9164a5db60943652643455a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:46.896526 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-76964af2-5008-4e88-ab15-3c4a04319d0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5230991c9164a5db60943652643455a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:46.908625 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-76964af2-5008-4e88-ab15-3c4a04319d0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5230991c9164a5db60943652643455a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:46.915598 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-76964af2-5008-4e88-ab15-3c4a04319d0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5230991c9164a5db60943652643455a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:46.924146 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-76964af2-5008-4e88-ab15-3c4a04319d0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5230991c9164a5db60943652643455a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:46.930697 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 981/1955] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:16:46 2022] DELETE /identity/v3/users/b5230991c9164a5db60943652643455a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:16:47.237559 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2b62e0-888b-42ce-9b1e-931c805e0d79 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:47.239065 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-7c2b62e0-888b-42ce-9b1e-931c805e0d79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:47.254172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2b62e0-888b-42ce-9b1e-931c805e0d79 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:47.319960 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ee9a0-b0f8-4891-9d22-111abf2a67e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:47.320836 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ee9a0-b0f8-4891-9d22-111abf2a67e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:47.321115 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ee9a0-b0f8-4891-9d22-111abf2a67e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:47.321389 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ee9a0-b0f8-4891-9d22-111abf2a67e3 admin admin] PATH_INFO: `/v3/projects/9ddb265f3807450991a434ef4210453b` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:47.322645 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59ee9a0-b0f8-4891-9d22-111abf2a67e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ddb265f3807450991a434ef4210453b)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:47.324661 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59ee9a0-b0f8-4891-9d22-111abf2a67e3 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:47.377514 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-f59ee9a0-b0f8-4891-9d22-111abf2a67e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ddb265f3807450991a434ef4210453b', 'request_id': 'req-f59ee9a0-b0f8-4891-9d22-111abf2a67e3'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:47.383625 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-f59ee9a0-b0f8-4891-9d22-111abf2a67e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ddb265f3807450991a434ef4210453b', 'request_id': 'req-f59ee9a0-b0f8-4891-9d22-111abf2a67e3'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:47.391498 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-f59ee9a0-b0f8-4891-9d22-111abf2a67e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ddb265f3807450991a434ef4210453b', 'request_id': 'req-f59ee9a0-b0f8-4891-9d22-111abf2a67e3'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:47.403571 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-f59ee9a0-b0f8-4891-9d22-111abf2a67e3 admin admin] The token cache is being invalidate because project 9ddb265f3807450991a434ef4210453b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:16:47.403867 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-f59ee9a0-b0f8-4891-9d22-111abf2a67e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:47.404510 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-f59ee9a0-b0f8-4891-9d22-111abf2a67e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:47.406079 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 975/1956] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:16:47 2022] DELETE /identity/v3/projects/9ddb265f3807450991a434ef4210453b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:16:47.411072 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb57df-c88a-45e2-b2ac-25055c7be5a9 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:47.411444 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb57df-c88a-45e2-b2ac-25055c7be5a9 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:47.411835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb57df-c88a-45e2-b2ac-25055c7be5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:47.430237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-67eb57df-c88a-45e2-b2ac-25055c7be5a9 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:16:47.431525 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-67eb57df-c88a-45e2-b2ac-25055c7be5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:47.476013 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:47.497961 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 982/1957] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:16:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:16:47.505714 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b002d03-3c58-4b09-adb5-ff51d7ca3354 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:47.506006 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b002d03-3c58-4b09-adb5-ff51d7ca3354 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:47.506289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b002d03-3c58-4b09-adb5-ff51d7ca3354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:47.525096 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-8b002d03-3c58-4b09-adb5-ff51d7ca3354 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:16:47.526812 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-8b002d03-3c58-4b09-adb5-ff51d7ca3354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:47.536676 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 976/1958] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:16:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:16:47.547411 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb57df-c88a-45e2-b2ac-25055c7be5a9 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:47.565410 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb57df-c88a-45e2-b2ac-25055c7be5a9 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:47.573248 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a883f545-3000-4ed8-b846-88d6670e6128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:47.574400 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883f545-3000-4ed8-b846-88d6670e6128 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:47.574884 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883f545-3000-4ed8-b846-88d6670e6128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:47.575368 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883f545-3000-4ed8-b846-88d6670e6128 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:47.576785 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a883f545-3000-4ed8-b846-88d6670e6128 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:16:47.577334 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a883f545-3000-4ed8-b846-88d6670e6128 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:47.580488 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a883f545-3000-4ed8-b846-88d6670e6128 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:47.596815 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 983/1959] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:16:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:16:47.603040 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b002d03-3c58-4b09-adb5-ff51d7ca3354 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:47.620215 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b002d03-3c58-4b09-adb5-ff51d7ca3354 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:47.628148 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5aad55-44fe-41bb-bf7f-411eeeee448f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:47.629274 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5aad55-44fe-41bb-bf7f-411eeeee448f admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:47.629769 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5aad55-44fe-41bb-bf7f-411eeeee448f admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:47.630257 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5aad55-44fe-41bb-bf7f-411eeeee448f admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:47.631527 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5aad55-44fe-41bb-bf7f-411eeeee448f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:47.635872 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5aad55-44fe-41bb-bf7f-411eeeee448f admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:47.657948 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a883f545-3000-4ed8-b846-88d6670e6128 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:47.660096 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a883f545-3000-4ed8-b846-88d6670e6128 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:47.684060 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 977/1960] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:16:47 2022] POST /identity/v3/projects => generated 379 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:16:47.685252 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5aad55-44fe-41bb-bf7f-411eeeee448f admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:47.753941 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a883f545-3000-4ed8-b846-88d6670e6128 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:47.778840 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5aad55-44fe-41bb-bf7f-411eeeee448f admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:47.785850 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b609bf00-794b-4e76-b054-dff3cb89c507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:47.786636 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b609bf00-794b-4e76-b054-dff3cb89c507 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:47.787029 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b609bf00-794b-4e76-b054-dff3cb89c507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:47.787485 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b609bf00-794b-4e76-b054-dff3cb89c507 admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:47.788414 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b609bf00-794b-4e76-b054-dff3cb89c507 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:47.791096 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b609bf00-794b-4e76-b054-dff3cb89c507 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:47.804152 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:47.836600 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 978/1961] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:16:47 2022] POST /identity/v3/users => generated 290 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:16:47.839889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad16ba-a577-4983-b40e-74d03a5fbf96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:47.841028 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad16ba-a577-4983-b40e-74d03a5fbf96 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:47.841481 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad16ba-a577-4983-b40e-74d03a5fbf96 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:47.841872 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b609bf00-794b-4e76-b054-dff3cb89c507 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:47.842071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad16ba-a577-4983-b40e-74d03a5fbf96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:47.844989 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-01ad16ba-a577-4983-b40e-74d03a5fbf96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:47.861152 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b609bf00-794b-4e76-b054-dff3cb89c507 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:47.869604 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d022161-d468-42fc-9828-3e1f2ca6d73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:47.870658 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d022161-d468-42fc-9828-3e1f2ca6d73d admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:47.871065 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d022161-d468-42fc-9828-3e1f2ca6d73d admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:47.871715 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d022161-d468-42fc-9828-3e1f2ca6d73d admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:47.872767 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d022161-d468-42fc-9828-3e1f2ca6d73d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:47.876545 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d022161-d468-42fc-9828-3e1f2ca6d73d admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:47.902502 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 979/1962] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:16:47.907981 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d022161-d468-42fc-9828-3e1f2ca6d73d admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:47.942352 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d022161-d468-42fc-9828-3e1f2ca6d73d admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:47.962866 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c8c11b-acb8-4cd1-9a04-18a1b48ff13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:47.965009 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8c11b-acb8-4cd1-9a04-18a1b48ff13c admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:47.965526 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8c11b-acb8-4cd1-9a04-18a1b48ff13c admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:47.966095 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8c11b-acb8-4cd1-9a04-18a1b48ff13c admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:47.967963 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c8c11b-acb8-4cd1-9a04-18a1b48ff13c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:47.972646 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c8c11b-acb8-4cd1-9a04-18a1b48ff13c admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:47.998075 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 980/1963] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:16:48.004408 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c8c11b-acb8-4cd1-9a04-18a1b48ff13c admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:48.030995 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ad16ba-a577-4983-b40e-74d03a5fbf96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:48.036211 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c8c11b-acb8-4cd1-9a04-18a1b48ff13c admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:48.051859 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ad16ba-a577-4983-b40e-74d03a5fbf96 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:48.061536 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 984/1964] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:16:47 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:16:48.061907 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b503ba3e-e993-42aa-9cf3-df07bfde5a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:48.064788 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503ba3e-e993-42aa-9cf3-df07bfde5a52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:48.065250 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503ba3e-e993-42aa-9cf3-df07bfde5a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:48.065945 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503ba3e-e993-42aa-9cf3-df07bfde5a52 admin admin] PATH_INFO: `/v3/projects/b372e8c27a86409ab7e2a0a43b0b5de7/users/947a4548687a417f87c5a8029601ec7f/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:48.095691 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b503ba3e-e993-42aa-9cf3-df07bfde5a52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b372e8c27a86409ab7e2a0a43b0b5de7, user_id=947a4548687a417f87c5a8029601ec7f, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:48.100698 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b503ba3e-e993-42aa-9cf3-df07bfde5a52 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:48.148776 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 981/1965] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:16:48 2022] PUT /identity/v3/projects/b372e8c27a86409ab7e2a0a43b0b5de7/users/947a4548687a417f87c5a8029601ec7f/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:16:48.157556 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677f0b6-0b78-48a1-b6f9-c09b0c8725d4 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:48.158176 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677f0b6-0b78-48a1-b6f9-c09b0c8725d4 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:48.159121 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677f0b6-0b78-48a1-b6f9-c09b0c8725d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:48.217715 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-b677f0b6-0b78-48a1-b6f9-c09b0c8725d4 None None] MFA Rules not processed for user `947a4548687a417f87c5a8029601ec7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:16:48.220693 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b677f0b6-0b78-48a1-b6f9-c09b0c8725d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:48.279121 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:48.478169 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 985/1966] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:16:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:16:48.486617 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b503ba3e-e993-42aa-9cf3-df07bfde5a52 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:48.719899 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b503ba3e-e993-42aa-9cf3-df07bfde5a52 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:48.743720 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0125f0d3-72d6-4eed-a05b-01061e72ad3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:48.743720 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0125f0d3-72d6-4eed-a05b-01061e72ad3e admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:48.743720 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0125f0d3-72d6-4eed-a05b-01061e72ad3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:48.743720 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0125f0d3-72d6-4eed-a05b-01061e72ad3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:48.752562 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0125f0d3-72d6-4eed-a05b-01061e72ad3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:48.752562 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0125f0d3-72d6-4eed-a05b-01061e72ad3e admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:48.832700 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 982/1967] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:16:48 2022] POST /identity/v3/projects => generated 377 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:16:48.837996 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b677f0b6-0b78-48a1-b6f9-c09b0c8725d4 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:48.864233 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b677f0b6-0b78-48a1-b6f9-c09b0c8725d4 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:48.996507 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bdb05a-7ee6-46cc-a6e3-b357059f3a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:48.997652 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bdb05a-7ee6-46cc-a6e3-b357059f3a9d admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:48.998112 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bdb05a-7ee6-46cc-a6e3-b357059f3a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:48.998558 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bdb05a-7ee6-46cc-a6e3-b357059f3a9d admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:48.999757 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bdb05a-7ee6-46cc-a6e3-b357059f3a9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:49.002960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bdb05a-7ee6-46cc-a6e3-b357059f3a9d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:49.051199 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 986/1968] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:16:48 2022] POST /identity/v3/users => generated 289 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:16:49.058479 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0125f0d3-72d6-4eed-a05b-01061e72ad3e admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:49.089291 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0125f0d3-72d6-4eed-a05b-01061e72ad3e admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:49.107019 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e653aa38-9219-45d1-9931-a55c66c78407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:49.108190 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e653aa38-9219-45d1-9931-a55c66c78407 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:49.108668 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e653aa38-9219-45d1-9931-a55c66c78407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:49.109161 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e653aa38-9219-45d1-9931-a55c66c78407 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:49.110572 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e653aa38-9219-45d1-9931-a55c66c78407 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:49.119756 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e653aa38-9219-45d1-9931-a55c66c78407 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:49.167664 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 983/1969] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:16:49.173942 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bdb05a-7ee6-46cc-a6e3-b357059f3a9d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:49.195053 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bdb05a-7ee6-46cc-a6e3-b357059f3a9d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:49.204932 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f947147-52cb-4df8-acb2-3a9df1ac8e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:49.206730 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f947147-52cb-4df8-acb2-3a9df1ac8e3a admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:49.206730 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f947147-52cb-4df8-acb2-3a9df1ac8e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:49.207285 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f947147-52cb-4df8-acb2-3a9df1ac8e3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:49.208859 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f947147-52cb-4df8-acb2-3a9df1ac8e3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:49.211339 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f947147-52cb-4df8-acb2-3a9df1ac8e3a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:49.229069 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 987/1970] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:16:49.230368 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e653aa38-9219-45d1-9931-a55c66c78407 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:49.248375 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e653aa38-9219-45d1-9931-a55c66c78407 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:49.257202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb37c4b-5201-4d70-8fb5-846a221614a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:49.258715 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb37c4b-5201-4d70-8fb5-846a221614a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:49.259572 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb37c4b-5201-4d70-8fb5-846a221614a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:49.259769 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb37c4b-5201-4d70-8fb5-846a221614a8 admin admin] PATH_INFO: `/v3/projects/d56018968ed64469aa8e5f5e1b6d257b/users/5d4876aa1b704946bb0e188379f0553a/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:49.290651 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb37c4b-5201-4d70-8fb5-846a221614a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d56018968ed64469aa8e5f5e1b6d257b, user_id=5d4876aa1b704946bb0e188379f0553a, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:49.296412 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb37c4b-5201-4d70-8fb5-846a221614a8 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:49.332238 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 984/1971] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:16:49 2022] PUT /identity/v3/projects/d56018968ed64469aa8e5f5e1b6d257b/users/5d4876aa1b704946bb0e188379f0553a/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:16:49.333314 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b982ac-a018-4813-8442-4ae9d16fb25b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:49.333607 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b982ac-a018-4813-8442-4ae9d16fb25b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:49.333886 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b982ac-a018-4813-8442-4ae9d16fb25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:49.371615 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-a8b982ac-a018-4813-8442-4ae9d16fb25b None None] MFA Rules not processed for user `5d4876aa1b704946bb0e188379f0553a`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:16:49.374523 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a8b982ac-a018-4813-8442-4ae9d16fb25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:49.469359 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 988/1972] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:16:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:16:49.474454 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb37c4b-5201-4d70-8fb5-846a221614a8 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:49.555930 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:49.564601 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb37c4b-5201-4d70-8fb5-846a221614a8 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:49.571340 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d89ccf-cdd4-4f2b-8147-838d2e031c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:49.571995 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d89ccf-cdd4-4f2b-8147-838d2e031c56 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:49.572277 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d89ccf-cdd4-4f2b-8147-838d2e031c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:49.572609 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d89ccf-cdd4-4f2b-8147-838d2e031c56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:49.574278 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d89ccf-cdd4-4f2b-8147-838d2e031c56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:49.576569 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d89ccf-cdd4-4f2b-8147-838d2e031c56 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:49.616653 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 985/1973] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:16:49 2022] POST /identity/v3/projects => generated 377 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:16:49.618462 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b982ac-a018-4813-8442-4ae9d16fb25b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:49.632906 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b982ac-a018-4813-8442-4ae9d16fb25b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:49.763934 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:49.858857 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2622be5b-cf04-4894-98a9-ab93c9741e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:49.860048 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622be5b-cf04-4894-98a9-ab93c9741e57 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:49.860536 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622be5b-cf04-4894-98a9-ab93c9741e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:49.860989 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622be5b-cf04-4894-98a9-ab93c9741e57 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:49.862410 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2622be5b-cf04-4894-98a9-ab93c9741e57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:49.866036 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2622be5b-cf04-4894-98a9-ab93c9741e57 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:49.896313 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 989/1974] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:16:49 2022] POST /identity/v3/users => generated 295 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:16:49.900221 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d89ccf-cdd4-4f2b-8147-838d2e031c56 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:49.911289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d89ccf-cdd4-4f2b-8147-838d2e031c56 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:49.917424 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda982b-41b4-45db-b55b-107683e76070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:49.918365 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda982b-41b4-45db-b55b-107683e76070 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:49.918747 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda982b-41b4-45db-b55b-107683e76070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:49.919095 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda982b-41b4-45db-b55b-107683e76070 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:49.920932 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cda982b-41b4-45db-b55b-107683e76070 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:49.923075 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cda982b-41b4-45db-b55b-107683e76070 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:49.941212 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 986/1975] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:16:49.942098 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2622be5b-cf04-4894-98a9-ab93c9741e57 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:49.953335 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2622be5b-cf04-4894-98a9-ab93c9741e57 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:49.958514 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af221fa-9a1c-412d-8b65-f9fbf6d9a8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:49.959545 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af221fa-9a1c-412d-8b65-f9fbf6d9a8c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:49.959971 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af221fa-9a1c-412d-8b65-f9fbf6d9a8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:49.960322 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af221fa-9a1c-412d-8b65-f9fbf6d9a8c0 admin admin] PATH_INFO: `/v3/projects/e9ce58ab1ba44d15a985ded223029409/users/710bb3a887c34904a38b0539b32db84d/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:49.981216 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af221fa-9a1c-412d-8b65-f9fbf6d9a8c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9ce58ab1ba44d15a985ded223029409, user_id=710bb3a887c34904a38b0539b32db84d, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:49.983972 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af221fa-9a1c-412d-8b65-f9fbf6d9a8c0 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:50.007240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 990/1976] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:16:49 2022] PUT /identity/v3/projects/e9ce58ab1ba44d15a985ded223029409/users/710bb3a887c34904a38b0539b32db84d/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:16:50.013189 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda982b-41b4-45db-b55b-107683e76070 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:50.030893 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda982b-41b4-45db-b55b-107683e76070 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:50.137643 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce9126-23b0-412a-81c9-aedc12c8f55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:50.138617 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce9126-23b0-412a-81c9-aedc12c8f55b admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:50.139443 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce9126-23b0-412a-81c9-aedc12c8f55b admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:50.139993 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce9126-23b0-412a-81c9-aedc12c8f55b admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:50.141012 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ce9126-23b0-412a-81c9-aedc12c8f55b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:50.142940 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ce9126-23b0-412a-81c9-aedc12c8f55b admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:50.159647 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 987/1977] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:16:50.164652 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af221fa-9a1c-412d-8b65-f9fbf6d9a8c0 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:50.181416 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af221fa-9a1c-412d-8b65-f9fbf6d9a8c0 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:50.193304 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de3e849-a3d9-4cc7-8dac-a7425e601f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:50.195366 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3e849-a3d9-4cc7-8dac-a7425e601f91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:50.195792 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3e849-a3d9-4cc7-8dac-a7425e601f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:50.195951 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3e849-a3d9-4cc7-8dac-a7425e601f91 admin admin] PATH_INFO: `/v3/projects/e9ce58ab1ba44d15a985ded223029409/users/710bb3a887c34904a38b0539b32db84d/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:50.205915 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de3e849-a3d9-4cc7-8dac-a7425e601f91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9ce58ab1ba44d15a985ded223029409, user_id=710bb3a887c34904a38b0539b32db84d, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:50.219604 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de3e849-a3d9-4cc7-8dac-a7425e601f91 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:50.247769 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 991/1978] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:16:50 2022] PUT /identity/v3/projects/e9ce58ab1ba44d15a985ded223029409/users/710bb3a887c34904a38b0539b32db84d/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:16:50.258730 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a03fb2-3cbd-4e8c-a353-0d2a46577434 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:50.259137 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a03fb2-3cbd-4e8c-a353-0d2a46577434 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:50.259494 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a03fb2-3cbd-4e8c-a353-0d2a46577434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:50.299567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-73a03fb2-3cbd-4e8c-a353-0d2a46577434 None None] MFA Rules not processed for user `710bb3a887c34904a38b0539b32db84d`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:16:50.301092 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-73a03fb2-3cbd-4e8c-a353-0d2a46577434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:50.406593 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 988/1979] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:16:50 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:16:50.420541 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de3e849-a3d9-4cc7-8dac-a7425e601f91 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:50.422583 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3de3e849-a3d9-4cc7-8dac-a7425e601f91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:50.489931 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de3e849-a3d9-4cc7-8dac-a7425e601f91 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:50.493847 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3b874-7ac0-4ff8-bc24-f795e537eb29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:50.494588 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3b874-7ac0-4ff8-bc24-f795e537eb29 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:50.494989 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3b874-7ac0-4ff8-bc24-f795e537eb29 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:50.495326 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3b874-7ac0-4ff8-bc24-f795e537eb29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:50.507018 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a3b874-7ac0-4ff8-bc24-f795e537eb29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:50.509171 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a3b874-7ac0-4ff8-bc24-f795e537eb29 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:50.522160 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 992/1980] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:16:50 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:16:50.567125 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a03fb2-3cbd-4e8c-a353-0d2a46577434 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:50.578300 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a03fb2-3cbd-4e8c-a353-0d2a46577434 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:50.582466 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513dcc1-d664-4ad2-89d8-33403ca7b6f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:50.583219 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513dcc1-d664-4ad2-89d8-33403ca7b6f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:50.583665 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513dcc1-d664-4ad2-89d8-33403ca7b6f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:50.583988 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513dcc1-d664-4ad2-89d8-33403ca7b6f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:50.643209 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:50.658928 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0513dcc1-d664-4ad2-89d8-33403ca7b6f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:50.660601 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0513dcc1-d664-4ad2-89d8-33403ca7b6f2 service cinder] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:50.671041 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 989/1981] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:16:50 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:16:53.951102 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3b874-7ac0-4ff8-bc24-f795e537eb29 service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:54.034783 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3b874-7ac0-4ff8-bc24-f795e537eb29 service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:54.043662 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89809794-d8c3-4a6b-86ef-7ec68a82dd93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:54.044809 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89809794-d8c3-4a6b-86ef-7ec68a82dd93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:54.045316 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89809794-d8c3-4a6b-86ef-7ec68a82dd93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:54.045814 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89809794-d8c3-4a6b-86ef-7ec68a82dd93 admin admin] PATH_INFO: `/v3/users/947a4548687a417f87c5a8029601ec7f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:54.048171 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89809794-d8c3-4a6b-86ef-7ec68a82dd93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=947a4548687a417f87c5a8029601ec7f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:54.051944 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89809794-d8c3-4a6b-86ef-7ec68a82dd93 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:54.104989 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:54.125496 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-89809794-d8c3-4a6b-86ef-7ec68a82dd93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '947a4548687a417f87c5a8029601ec7f'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:54.130400 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-89809794-d8c3-4a6b-86ef-7ec68a82dd93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '947a4548687a417f87c5a8029601ec7f'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:54.157028 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-89809794-d8c3-4a6b-86ef-7ec68a82dd93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '947a4548687a417f87c5a8029601ec7f'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:54.172048 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-89809794-d8c3-4a6b-86ef-7ec68a82dd93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '947a4548687a417f87c5a8029601ec7f'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:54.185479 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-89809794-d8c3-4a6b-86ef-7ec68a82dd93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '947a4548687a417f87c5a8029601ec7f'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:54.194677 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 993/1982] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:16:53 2022] DELETE /identity/v3/users/947a4548687a417f87c5a8029601ec7f => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:16:54.199712 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085a3d7-7aa7-41d4-aee3-240238696c97 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:54.200508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085a3d7-7aa7-41d4-aee3-240238696c97 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:54.200665 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085a3d7-7aa7-41d4-aee3-240238696c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:54.220354 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-a085a3d7-7aa7-41d4-aee3-240238696c97 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:16:54.221701 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-a085a3d7-7aa7-41d4-aee3-240238696c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:54.298816 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 990/1983] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:16:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:16:54.309192 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89809794-d8c3-4a6b-86ef-7ec68a82dd93 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:54.313872 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-89809794-d8c3-4a6b-86ef-7ec68a82dd93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:54.431119 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89809794-d8c3-4a6b-86ef-7ec68a82dd93 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:54.437164 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520bd38-ecab-4d08-83dc-5c8beefd1ba0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:54.438193 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520bd38-ecab-4d08-83dc-5c8beefd1ba0 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:54.438752 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520bd38-ecab-4d08-83dc-5c8beefd1ba0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:54.439159 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520bd38-ecab-4d08-83dc-5c8beefd1ba0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:54.465177 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3520bd38-ecab-4d08-83dc-5c8beefd1ba0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:54.468524 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3520bd38-ecab-4d08-83dc-5c8beefd1ba0 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:54.485153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 994/1984] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:16:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:16:54.848868 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a085a3d7-7aa7-41d4-aee3-240238696c97 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:54.850773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-a085a3d7-7aa7-41d4-aee3-240238696c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:54.866225 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a085a3d7-7aa7-41d4-aee3-240238696c97 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:54.872152 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a6c0d2-2ce6-4006-a468-b7f9c19d3789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:54.873348 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6c0d2-2ce6-4006-a468-b7f9c19d3789 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:54.873524 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6c0d2-2ce6-4006-a468-b7f9c19d3789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:54.873788 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6c0d2-2ce6-4006-a468-b7f9c19d3789 admin admin] PATH_INFO: `/v3/projects/b372e8c27a86409ab7e2a0a43b0b5de7` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:54.875158 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a6c0d2-2ce6-4006-a468-b7f9c19d3789 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b372e8c27a86409ab7e2a0a43b0b5de7)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:54.877377 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a6c0d2-2ce6-4006-a468-b7f9c19d3789 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:54.936901 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-15a6c0d2-2ce6-4006-a468-b7f9c19d3789 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b372e8c27a86409ab7e2a0a43b0b5de7', 'request_id': 'req-15a6c0d2-2ce6-4006-a468-b7f9c19d3789'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:54.943908 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-15a6c0d2-2ce6-4006-a468-b7f9c19d3789 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b372e8c27a86409ab7e2a0a43b0b5de7', 'request_id': 'req-15a6c0d2-2ce6-4006-a468-b7f9c19d3789'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:54.952281 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-15a6c0d2-2ce6-4006-a468-b7f9c19d3789 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b372e8c27a86409ab7e2a0a43b0b5de7', 'request_id': 'req-15a6c0d2-2ce6-4006-a468-b7f9c19d3789'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:54.967221 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-15a6c0d2-2ce6-4006-a468-b7f9c19d3789 admin admin] The token cache is being invalidate because project b372e8c27a86409ab7e2a0a43b0b5de7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:16:54.967221 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-15a6c0d2-2ce6-4006-a468-b7f9c19d3789 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:54.968131 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-15a6c0d2-2ce6-4006-a468-b7f9c19d3789 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:54.969953 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 991/1985] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:16:54 2022] DELETE /identity/v3/projects/b372e8c27a86409ab7e2a0a43b0b5de7 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:16:54.974533 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520bd38-ecab-4d08-83dc-5c8beefd1ba0 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:54.985852 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520bd38-ecab-4d08-83dc-5c8beefd1ba0 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:55.083254 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f588e2-7860-416c-8c5d-bf9722696bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:55.084312 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f588e2-7860-416c-8c5d-bf9722696bae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:55.084772 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f588e2-7860-416c-8c5d-bf9722696bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:55.085149 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f588e2-7860-416c-8c5d-bf9722696bae admin admin] PATH_INFO: `/v3/users/5d4876aa1b704946bb0e188379f0553a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:55.086651 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f588e2-7860-416c-8c5d-bf9722696bae admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d4876aa1b704946bb0e188379f0553a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:55.089564 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f588e2-7860-416c-8c5d-bf9722696bae admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:55.164476 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d9f588e2-7860-416c-8c5d-bf9722696bae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d4876aa1b704946bb0e188379f0553a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:55.165468 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d9f588e2-7860-416c-8c5d-bf9722696bae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d4876aa1b704946bb0e188379f0553a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:55.176439 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:55.192228 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d9f588e2-7860-416c-8c5d-bf9722696bae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d4876aa1b704946bb0e188379f0553a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:55.204352 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d9f588e2-7860-416c-8c5d-bf9722696bae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d4876aa1b704946bb0e188379f0553a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:55.218543 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d9f588e2-7860-416c-8c5d-bf9722696bae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d4876aa1b704946bb0e188379f0553a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:55.232405 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 995/1986] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:16:54 2022] DELETE /identity/v3/users/5d4876aa1b704946bb0e188379f0553a => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:16:55.616315 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a6c0d2-2ce6-4006-a468-b7f9c19d3789 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:55.618499 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-15a6c0d2-2ce6-4006-a468-b7f9c19d3789 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:55.642793 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a6c0d2-2ce6-4006-a468-b7f9c19d3789 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:55.746951 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbaf19c-a149-4049-8f8d-e884831898e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:55.747921 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbaf19c-a149-4049-8f8d-e884831898e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:55.748209 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbaf19c-a149-4049-8f8d-e884831898e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:55.748934 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbaf19c-a149-4049-8f8d-e884831898e0 admin admin] PATH_INFO: `/v3/projects/d56018968ed64469aa8e5f5e1b6d257b` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:55.750260 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbaf19c-a149-4049-8f8d-e884831898e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d56018968ed64469aa8e5f5e1b6d257b)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:55.752358 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbaf19c-a149-4049-8f8d-e884831898e0 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:55.774201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:55.810775 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-6cbaf19c-a149-4049-8f8d-e884831898e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd56018968ed64469aa8e5f5e1b6d257b', 'request_id': 'req-6cbaf19c-a149-4049-8f8d-e884831898e0'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:55.819916 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-6cbaf19c-a149-4049-8f8d-e884831898e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd56018968ed64469aa8e5f5e1b6d257b', 'request_id': 'req-6cbaf19c-a149-4049-8f8d-e884831898e0'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:55.830332 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-6cbaf19c-a149-4049-8f8d-e884831898e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd56018968ed64469aa8e5f5e1b6d257b', 'request_id': 'req-6cbaf19c-a149-4049-8f8d-e884831898e0'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:55.847162 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-6cbaf19c-a149-4049-8f8d-e884831898e0 admin admin] The token cache is being invalidate because project d56018968ed64469aa8e5f5e1b6d257b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:16:55.847519 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-6cbaf19c-a149-4049-8f8d-e884831898e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:55.848322 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-6cbaf19c-a149-4049-8f8d-e884831898e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:55.849975 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 992/1987] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:16:55 2022] DELETE /identity/v3/projects/d56018968ed64469aa8e5f5e1b6d257b => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:16:55.855444 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f588e2-7860-416c-8c5d-bf9722696bae admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:55.857646 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d9f588e2-7860-416c-8c5d-bf9722696bae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:55.877663 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f588e2-7860-416c-8c5d-bf9722696bae admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:56.017935 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c565fe2-6330-4d7b-b2be-bbad92d6c3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:56.019045 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c565fe2-6330-4d7b-b2be-bbad92d6c3c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:56.019488 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c565fe2-6330-4d7b-b2be-bbad92d6c3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:56.020070 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c565fe2-6330-4d7b-b2be-bbad92d6c3c7 admin admin] PATH_INFO: `/v3/users/710bb3a887c34904a38b0539b32db84d` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:56.022351 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c565fe2-6330-4d7b-b2be-bbad92d6c3c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=710bb3a887c34904a38b0539b32db84d)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:56.026223 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c565fe2-6330-4d7b-b2be-bbad92d6c3c7 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:56.104189 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0c565fe2-6330-4d7b-b2be-bbad92d6c3c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '710bb3a887c34904a38b0539b32db84d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:56.105136 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0c565fe2-6330-4d7b-b2be-bbad92d6c3c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '710bb3a887c34904a38b0539b32db84d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:56.127130 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0c565fe2-6330-4d7b-b2be-bbad92d6c3c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '710bb3a887c34904a38b0539b32db84d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:56.142613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0c565fe2-6330-4d7b-b2be-bbad92d6c3c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '710bb3a887c34904a38b0539b32db84d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:56.158282 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0c565fe2-6330-4d7b-b2be-bbad92d6c3c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '710bb3a887c34904a38b0539b32db84d'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:56.189333 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 996/1988] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:16:55 2022] DELETE /identity/v3/users/710bb3a887c34904a38b0539b32db84d => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:16:56.601522 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbaf19c-a149-4049-8f8d-e884831898e0 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:56.604647 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-6cbaf19c-a149-4049-8f8d-e884831898e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:56.636069 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbaf19c-a149-4049-8f8d-e884831898e0 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:56.719090 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b045920-055d-451c-981f-b4617bff8642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:56.719937 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b045920-055d-451c-981f-b4617bff8642 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:56.720259 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b045920-055d-451c-981f-b4617bff8642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:56.720540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b045920-055d-451c-981f-b4617bff8642 admin admin] PATH_INFO: `/v3/projects/e9ce58ab1ba44d15a985ded223029409` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:56.722064 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b045920-055d-451c-981f-b4617bff8642 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9ce58ab1ba44d15a985ded223029409)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:56.724177 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b045920-055d-451c-981f-b4617bff8642 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:56.781168 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-6b045920-055d-451c-981f-b4617bff8642 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9ce58ab1ba44d15a985ded223029409', 'request_id': 'req-6b045920-055d-451c-981f-b4617bff8642'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:56.788933 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-6b045920-055d-451c-981f-b4617bff8642 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9ce58ab1ba44d15a985ded223029409', 'request_id': 'req-6b045920-055d-451c-981f-b4617bff8642'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:56.798167 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-6b045920-055d-451c-981f-b4617bff8642 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9ce58ab1ba44d15a985ded223029409', 'request_id': 'req-6b045920-055d-451c-981f-b4617bff8642'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:56.811210 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-6b045920-055d-451c-981f-b4617bff8642 admin admin] The token cache is being invalidate because project e9ce58ab1ba44d15a985ded223029409 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:16:56.811612 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-6b045920-055d-451c-981f-b4617bff8642 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:56.812206 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-6b045920-055d-451c-981f-b4617bff8642 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:56.813865 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 993/1989] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:16:56 2022] DELETE /identity/v3/projects/e9ce58ab1ba44d15a985ded223029409 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:16:56.820205 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac5e0b-a4d3-43f3-ad3c-2e5cd4283008 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:56.820622 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac5e0b-a4d3-43f3-ad3c-2e5cd4283008 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:56.820814 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac5e0b-a4d3-43f3-ad3c-2e5cd4283008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:56.841627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-1fac5e0b-a4d3-43f3-ad3c-2e5cd4283008 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:16:56.842875 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1fac5e0b-a4d3-43f3-ad3c-2e5cd4283008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:56.896157 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:56.930138 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 997/1990] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:16:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:16:56.932285 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18122ae-d903-4e3b-96fd-e052e1ac6b9a None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:56.932646 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18122ae-d903-4e3b-96fd-e052e1ac6b9a None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:56.932898 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18122ae-d903-4e3b-96fd-e052e1ac6b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:56.956326 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-a18122ae-d903-4e3b-96fd-e052e1ac6b9a None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:16:56.957591 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-a18122ae-d903-4e3b-96fd-e052e1ac6b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:56.967061 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 994/1991] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:16:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:16:56.971954 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac5e0b-a4d3-43f3-ad3c-2e5cd4283008 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:56.984089 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac5e0b-a4d3-43f3-ad3c-2e5cd4283008 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:56.991767 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdbc1a3-437e-48ba-b0e2-0c180919f2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:56.992706 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbc1a3-437e-48ba-b0e2-0c180919f2c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:56.993348 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbc1a3-437e-48ba-b0e2-0c180919f2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:56.994206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbc1a3-437e-48ba-b0e2-0c180919f2c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:56.996140 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdbc1a3-437e-48ba-b0e2-0c180919f2c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:16:56.997700 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdbc1a3-437e-48ba-b0e2-0c180919f2c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:56.999837 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdbc1a3-437e-48ba-b0e2-0c180919f2c6 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:57.021694 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 998/1992] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:16:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:16:57.024680 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18122ae-d903-4e3b-96fd-e052e1ac6b9a None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:57.051571 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18122ae-d903-4e3b-96fd-e052e1ac6b9a None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:57.059297 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467079a-cae1-4058-bc12-bbbb3365102d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:57.060426 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467079a-cae1-4058-bc12-bbbb3365102d admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:57.060820 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467079a-cae1-4058-bc12-bbbb3365102d admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:57.061188 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467079a-cae1-4058-bc12-bbbb3365102d admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:57.062087 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b467079a-cae1-4058-bc12-bbbb3365102d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:57.064687 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b467079a-cae1-4058-bc12-bbbb3365102d admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:57.098642 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 995/1993] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:16:57 2022] POST /identity/v3/projects => generated 379 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:16:57.101572 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdbc1a3-437e-48ba-b0e2-0c180919f2c6 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:57.113408 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdbc1a3-437e-48ba-b0e2-0c180919f2c6 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:57.191427 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d6649b-8929-43a4-adbf-32749ecd0b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:57.192452 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6649b-8929-43a4-adbf-32749ecd0b48 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:57.192851 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6649b-8929-43a4-adbf-32749ecd0b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:57.193242 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6649b-8929-43a4-adbf-32749ecd0b48 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:57.194462 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d6649b-8929-43a4-adbf-32749ecd0b48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:57.196997 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d6649b-8929-43a4-adbf-32749ecd0b48 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:57.232576 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 999/1994] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:16:57 2022] POST /identity/v3/users => generated 290 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:16:57.236556 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467079a-cae1-4058-bc12-bbbb3365102d admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:57.245057 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:57.251366 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467079a-cae1-4058-bc12-bbbb3365102d admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:57.258548 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfd5a5f-631c-49de-95a2-524349407531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:57.259573 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd5a5f-631c-49de-95a2-524349407531 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:57.261091 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd5a5f-631c-49de-95a2-524349407531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:57.261688 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd5a5f-631c-49de-95a2-524349407531 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:57.262832 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfd5a5f-631c-49de-95a2-524349407531 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:57.265405 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfd5a5f-631c-49de-95a2-524349407531 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:57.279041 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 996/1995] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:16:57.283799 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d6649b-8929-43a4-adbf-32749ecd0b48 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:57.297328 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d6649b-8929-43a4-adbf-32749ecd0b48 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:57.302822 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8faa8f-57fd-4a76-ab64-4a89d45494fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:57.303526 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8faa8f-57fd-4a76-ab64-4a89d45494fb admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:57.303835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8faa8f-57fd-4a76-ab64-4a89d45494fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:57.304377 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8faa8f-57fd-4a76-ab64-4a89d45494fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:57.305143 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8faa8f-57fd-4a76-ab64-4a89d45494fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:57.307385 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8faa8f-57fd-4a76-ab64-4a89d45494fb admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:57.319616 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1000/1996] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:16:57.324909 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfd5a5f-631c-49de-95a2-524349407531 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:57.337730 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfd5a5f-631c-49de-95a2-524349407531 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:57.343213 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9750f88-c14c-439b-9f30-2415f7e60dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:57.344260 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9750f88-c14c-439b-9f30-2415f7e60dd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:57.344540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9750f88-c14c-439b-9f30-2415f7e60dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:57.344816 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9750f88-c14c-439b-9f30-2415f7e60dd7 admin admin] PATH_INFO: `/v3/projects/29cbc0f207634b90ad0ff369c5a91ca6/users/942ceafd8dd84a71accba563c27bc930/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:57.364099 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9750f88-c14c-439b-9f30-2415f7e60dd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29cbc0f207634b90ad0ff369c5a91ca6, user_id=942ceafd8dd84a71accba563c27bc930, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:57.369800 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9750f88-c14c-439b-9f30-2415f7e60dd7 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:57.387996 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 997/1997] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:16:57 2022] PUT /identity/v3/projects/29cbc0f207634b90ad0ff369c5a91ca6/users/942ceafd8dd84a71accba563c27bc930/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:16:57.401124 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f0eab-b9b8-496b-b627-1999b85de2df None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:57.401881 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f0eab-b9b8-496b-b627-1999b85de2df None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:57.402418 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f0eab-b9b8-496b-b627-1999b85de2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:57.479943 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-d28f0eab-b9b8-496b-b627-1999b85de2df None None] MFA Rules not processed for user `942ceafd8dd84a71accba563c27bc930`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:16:57.482687 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d28f0eab-b9b8-496b-b627-1999b85de2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:57.637818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1001/1998] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:16:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:16:57.643010 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9750f88-c14c-439b-9f30-2415f7e60dd7 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:57.764799 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9750f88-c14c-439b-9f30-2415f7e60dd7 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:57.771801 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d991fb-05e1-479b-a996-b4e779e0e2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:57.772740 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d991fb-05e1-479b-a996-b4e779e0e2db admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:57.773015 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d991fb-05e1-479b-a996-b4e779e0e2db admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:57.773375 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d991fb-05e1-479b-a996-b4e779e0e2db admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:57.774096 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d991fb-05e1-479b-a996-b4e779e0e2db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:57.776746 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d991fb-05e1-479b-a996-b4e779e0e2db admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:57.815642 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 998/1999] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:16:57 2022] POST /identity/v3/projects => generated 381 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:16:57.819583 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f0eab-b9b8-496b-b627-1999b85de2df None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:57.832014 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f0eab-b9b8-496b-b627-1999b85de2df None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:57.897236 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:57.924720 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f82b339-658b-415a-a31d-03e9e5bb141a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:57.925587 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82b339-658b-415a-a31d-03e9e5bb141a admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:57.926057 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82b339-658b-415a-a31d-03e9e5bb141a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:57.926457 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f82b339-658b-415a-a31d-03e9e5bb141a admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:57.928400 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f82b339-658b-415a-a31d-03e9e5bb141a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:57.931041 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f82b339-658b-415a-a31d-03e9e5bb141a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:57.955251 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1002/2000] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:16:57 2022] POST /identity/v3/users => generated 297 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:16:57.959613 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d991fb-05e1-479b-a996-b4e779e0e2db admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:57.972966 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d991fb-05e1-479b-a996-b4e779e0e2db admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:57.978110 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d89a6a-7579-4a2b-9617-c97753c87f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:57.979746 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d89a6a-7579-4a2b-9617-c97753c87f88 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:57.980033 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d89a6a-7579-4a2b-9617-c97753c87f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:57.980309 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d89a6a-7579-4a2b-9617-c97753c87f88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:57.981102 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d89a6a-7579-4a2b-9617-c97753c87f88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:57.982876 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d89a6a-7579-4a2b-9617-c97753c87f88 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:58.001875 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 999/2001] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:16:58.003181 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f82b339-658b-415a-a31d-03e9e5bb141a admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:58.018176 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f82b339-658b-415a-a31d-03e9e5bb141a admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:58.027215 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f2e2ce-56cc-4929-974b-66b556167fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:58.028870 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2e2ce-56cc-4929-974b-66b556167fbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:58.029717 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2e2ce-56cc-4929-974b-66b556167fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:58.030644 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2e2ce-56cc-4929-974b-66b556167fbb admin admin] PATH_INFO: `/v3/projects/3c9a1558a1f94cbfa9f98eb28d1b02c1/users/42b7891a3fc44a5498ff47c85c9376ba/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:58.059979 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f2e2ce-56cc-4929-974b-66b556167fbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c9a1558a1f94cbfa9f98eb28d1b02c1, user_id=42b7891a3fc44a5498ff47c85c9376ba, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:58.065828 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f2e2ce-56cc-4929-974b-66b556167fbb admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:58.087134 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1003/2002] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:16:58 2022] PUT /identity/v3/projects/3c9a1558a1f94cbfa9f98eb28d1b02c1/users/42b7891a3fc44a5498ff47c85c9376ba/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:16:58.092395 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d89a6a-7579-4a2b-9617-c97753c87f88 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:58.106323 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d89a6a-7579-4a2b-9617-c97753c87f88 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:58.222936 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49da151c-6ff7-4792-a9b4-254dd27de12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:58.224564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da151c-6ff7-4792-a9b4-254dd27de12a admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:58.224756 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da151c-6ff7-4792-a9b4-254dd27de12a admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:58.225893 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49da151c-6ff7-4792-a9b4-254dd27de12a admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:58.226755 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49da151c-6ff7-4792-a9b4-254dd27de12a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:58.230127 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49da151c-6ff7-4792-a9b4-254dd27de12a admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:58.264280 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1000/2003] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:16:58 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:16:58.270296 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f2e2ce-56cc-4929-974b-66b556167fbb admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:58.291105 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f2e2ce-56cc-4929-974b-66b556167fbb admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:58.302161 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f8177-208e-44ba-bfbd-a25ce39d2903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:58.305895 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f8177-208e-44ba-bfbd-a25ce39d2903 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:58.307985 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f8177-208e-44ba-bfbd-a25ce39d2903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:58.309007 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f8177-208e-44ba-bfbd-a25ce39d2903 admin admin] PATH_INFO: `/v3/projects/3c9a1558a1f94cbfa9f98eb28d1b02c1/users/42b7891a3fc44a5498ff47c85c9376ba/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:58.313166 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9f8177-208e-44ba-bfbd-a25ce39d2903 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c9a1558a1f94cbfa9f98eb28d1b02c1, user_id=42b7891a3fc44a5498ff47c85c9376ba, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:58.319507 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9f8177-208e-44ba-bfbd-a25ce39d2903 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:58.345158 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1004/2004] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:16:58 2022] PUT /identity/v3/projects/3c9a1558a1f94cbfa9f98eb28d1b02c1/users/42b7891a3fc44a5498ff47c85c9376ba/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:16:58.365205 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21d7f3-4044-4d11-bf06-c1d87f59519c None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:58.365841 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21d7f3-4044-4d11-bf06-c1d87f59519c None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:58.366435 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21d7f3-4044-4d11-bf06-c1d87f59519c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:58.420603 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-fa21d7f3-4044-4d11-bf06-c1d87f59519c None None] MFA Rules not processed for user `42b7891a3fc44a5498ff47c85c9376ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:16:58.423717 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-fa21d7f3-4044-4d11-bf06-c1d87f59519c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:58.499375 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:16:58.590150 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1001/2005] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:16:58 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:16:58.620197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f8177-208e-44ba-bfbd-a25ce39d2903 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:58.626343 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ff9f8177-208e-44ba-bfbd-a25ce39d2903 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:58.823814 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f8177-208e-44ba-bfbd-a25ce39d2903 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:58.831878 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36feaebe-3f23-471a-9250-680f7719bb85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:58.833091 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36feaebe-3f23-471a-9250-680f7719bb85 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:58.833837 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36feaebe-3f23-471a-9250-680f7719bb85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:58.834168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36feaebe-3f23-471a-9250-680f7719bb85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:58.859523 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36feaebe-3f23-471a-9250-680f7719bb85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:58.864477 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36feaebe-3f23-471a-9250-680f7719bb85 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:58.905657 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1005/2006] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:16:58 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:16:59.580149 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21d7f3-4044-4d11-bf06-c1d87f59519c None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:59.680357 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21d7f3-4044-4d11-bf06-c1d87f59519c None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:16:59.685189 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e0934f-f94f-446a-8b85-a54ccbe4d2ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:16:59.686114 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0934f-f94f-446a-8b85-a54ccbe4d2ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:59.686413 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0934f-f94f-446a-8b85-a54ccbe4d2ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:59.686670 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e0934f-f94f-446a-8b85-a54ccbe4d2ab admin admin] PATH_INFO: `/v3/users/942ceafd8dd84a71accba563c27bc930` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:59.688702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e0934f-f94f-446a-8b85-a54ccbe4d2ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=942ceafd8dd84a71accba563c27bc930)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:16:59.690482 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e0934f-f94f-446a-8b85-a54ccbe4d2ab admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:16:59.763363 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e1e0934f-f94f-446a-8b85-a54ccbe4d2ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '942ceafd8dd84a71accba563c27bc930'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:59.764146 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e1e0934f-f94f-446a-8b85-a54ccbe4d2ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '942ceafd8dd84a71accba563c27bc930'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:59.784055 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e1e0934f-f94f-446a-8b85-a54ccbe4d2ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '942ceafd8dd84a71accba563c27bc930'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:59.799974 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e1e0934f-f94f-446a-8b85-a54ccbe4d2ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '942ceafd8dd84a71accba563c27bc930'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:59.801334 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e1e0934f-f94f-446a-8b85-a54ccbe4d2ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '942ceafd8dd84a71accba563c27bc930'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:16:59.825726 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1002/2007] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:16:59 2022] DELETE /identity/v3/users/942ceafd8dd84a71accba563c27bc930 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:16:59.831448 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3dc74-efdb-497d-8a36-51a55004cf04 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:16:59.832092 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3dc74-efdb-497d-8a36-51a55004cf04 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:16:59.832092 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3dc74-efdb-497d-8a36-51a55004cf04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:16:59.854539 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-bac3dc74-efdb-497d-8a36-51a55004cf04 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:16:59.855894 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-bac3dc74-efdb-497d-8a36-51a55004cf04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:16:59.952825 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1006/2008] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:16:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:16:59.966322 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e0934f-f94f-446a-8b85-a54ccbe4d2ab admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:16:59.968759 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-e1e0934f-f94f-446a-8b85-a54ccbe4d2ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:00.064475 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:00.090693 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e0934f-f94f-446a-8b85-a54ccbe4d2ab admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:00.099446 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd547c6-8913-4e2c-9cce-c79c2ae0bbd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:00.100246 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd547c6-8913-4e2c-9cce-c79c2ae0bbd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:00.100573 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd547c6-8913-4e2c-9cce-c79c2ae0bbd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:00.101559 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd547c6-8913-4e2c-9cce-c79c2ae0bbd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:00.113124 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd547c6-8913-4e2c-9cce-c79c2ae0bbd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:00.114567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd547c6-8913-4e2c-9cce-c79c2ae0bbd9 service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:00.128171 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1003/2009] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:16:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:17:00.613059 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3dc74-efdb-497d-8a36-51a55004cf04 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:00.615327 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-bac3dc74-efdb-497d-8a36-51a55004cf04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:00.654810 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3dc74-efdb-497d-8a36-51a55004cf04 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:00.665746 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8513c8db-d4ff-436c-9651-144e22f1cbc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:00.667627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8513c8db-d4ff-436c-9651-144e22f1cbc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:00.668331 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8513c8db-d4ff-436c-9651-144e22f1cbc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:00.668825 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8513c8db-d4ff-436c-9651-144e22f1cbc5 admin admin] PATH_INFO: `/v3/projects/29cbc0f207634b90ad0ff369c5a91ca6` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:00.671714 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8513c8db-d4ff-436c-9651-144e22f1cbc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29cbc0f207634b90ad0ff369c5a91ca6)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:00.675630 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8513c8db-d4ff-436c-9651-144e22f1cbc5 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:00.790672 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:00.867330 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8513c8db-d4ff-436c-9651-144e22f1cbc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29cbc0f207634b90ad0ff369c5a91ca6', 'request_id': 'req-8513c8db-d4ff-436c-9651-144e22f1cbc5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:00.889036 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8513c8db-d4ff-436c-9651-144e22f1cbc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29cbc0f207634b90ad0ff369c5a91ca6', 'request_id': 'req-8513c8db-d4ff-436c-9651-144e22f1cbc5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:00.909064 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8513c8db-d4ff-436c-9651-144e22f1cbc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29cbc0f207634b90ad0ff369c5a91ca6', 'request_id': 'req-8513c8db-d4ff-436c-9651-144e22f1cbc5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:00.925585 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8513c8db-d4ff-436c-9651-144e22f1cbc5 admin admin] The token cache is being invalidate because project 29cbc0f207634b90ad0ff369c5a91ca6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:17:00.926364 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8513c8db-d4ff-436c-9651-144e22f1cbc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:00.927494 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8513c8db-d4ff-436c-9651-144e22f1cbc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:00.930279 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1007/2010] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:17:00 2022] DELETE /identity/v3/projects/29cbc0f207634b90ad0ff369c5a91ca6 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:00.936675 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd547c6-8913-4e2c-9cce-c79c2ae0bbd9 service neutron] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:00.950987 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd547c6-8913-4e2c-9cce-c79c2ae0bbd9 service neutron] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:01.042817 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e065c9c1-256d-4e80-86ad-804f3b09dbf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:01.044307 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e065c9c1-256d-4e80-86ad-804f3b09dbf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:01.044601 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e065c9c1-256d-4e80-86ad-804f3b09dbf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:01.044873 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e065c9c1-256d-4e80-86ad-804f3b09dbf4 admin admin] PATH_INFO: `/v3/users/42b7891a3fc44a5498ff47c85c9376ba` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:01.047618 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e065c9c1-256d-4e80-86ad-804f3b09dbf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42b7891a3fc44a5498ff47c85c9376ba)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:01.049881 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e065c9c1-256d-4e80-86ad-804f3b09dbf4 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:01.161036 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e065c9c1-256d-4e80-86ad-804f3b09dbf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42b7891a3fc44a5498ff47c85c9376ba'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:01.162439 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e065c9c1-256d-4e80-86ad-804f3b09dbf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42b7891a3fc44a5498ff47c85c9376ba'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:01.225952 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e065c9c1-256d-4e80-86ad-804f3b09dbf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42b7891a3fc44a5498ff47c85c9376ba'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:01.266066 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e065c9c1-256d-4e80-86ad-804f3b09dbf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42b7891a3fc44a5498ff47c85c9376ba'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:01.273192 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e065c9c1-256d-4e80-86ad-804f3b09dbf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42b7891a3fc44a5498ff47c85c9376ba'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:01.316515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1004/2011] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:17:00 2022] DELETE /identity/v3/users/42b7891a3fc44a5498ff47c85c9376ba => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:01.936801 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8513c8db-d4ff-436c-9651-144e22f1cbc5 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:01.939289 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8513c8db-d4ff-436c-9651-144e22f1cbc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:01.974314 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8513c8db-d4ff-436c-9651-144e22f1cbc5 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:02.122132 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a69655-3936-4016-b0be-dfaf988d9a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:02.123042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a69655-3936-4016-b0be-dfaf988d9a6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:02.123669 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a69655-3936-4016-b0be-dfaf988d9a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:02.123956 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a69655-3936-4016-b0be-dfaf988d9a6c admin admin] PATH_INFO: `/v3/projects/3c9a1558a1f94cbfa9f98eb28d1b02c1` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:02.126455 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a69655-3936-4016-b0be-dfaf988d9a6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c9a1558a1f94cbfa9f98eb28d1b02c1)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:02.128322 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a69655-3936-4016-b0be-dfaf988d9a6c admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:02.191690 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c7a69655-3936-4016-b0be-dfaf988d9a6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c9a1558a1f94cbfa9f98eb28d1b02c1', 'request_id': 'req-c7a69655-3936-4016-b0be-dfaf988d9a6c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:02.200668 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c7a69655-3936-4016-b0be-dfaf988d9a6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c9a1558a1f94cbfa9f98eb28d1b02c1', 'request_id': 'req-c7a69655-3936-4016-b0be-dfaf988d9a6c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:02.210558 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c7a69655-3936-4016-b0be-dfaf988d9a6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c9a1558a1f94cbfa9f98eb28d1b02c1', 'request_id': 'req-c7a69655-3936-4016-b0be-dfaf988d9a6c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:02.231211 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c7a69655-3936-4016-b0be-dfaf988d9a6c admin admin] The token cache is being invalidate because project 3c9a1558a1f94cbfa9f98eb28d1b02c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:17:02.231769 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c7a69655-3936-4016-b0be-dfaf988d9a6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:02.232806 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c7a69655-3936-4016-b0be-dfaf988d9a6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:02.236048 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1008/2012] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:17:01 2022] DELETE /identity/v3/projects/3c9a1558a1f94cbfa9f98eb28d1b02c1 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:02.241906 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91144ce7-cb93-425f-a024-ee25e3b19169 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:02.242618 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91144ce7-cb93-425f-a024-ee25e3b19169 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:02.242882 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91144ce7-cb93-425f-a024-ee25e3b19169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:02.274652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-91144ce7-cb93-425f-a024-ee25e3b19169 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:02.277381 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-91144ce7-cb93-425f-a024-ee25e3b19169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:02.401905 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1005/2013] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:02.412248 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33336b94-0298-43b3-bbaa-1350401af083 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:02.412866 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33336b94-0298-43b3-bbaa-1350401af083 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:02.413493 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33336b94-0298-43b3-bbaa-1350401af083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:02.468089 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-33336b94-0298-43b3-bbaa-1350401af083 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:02.470610 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-33336b94-0298-43b3-bbaa-1350401af083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:02.506687 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1009/2014] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:02.512582 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91144ce7-cb93-425f-a024-ee25e3b19169 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:02.530146 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91144ce7-cb93-425f-a024-ee25e3b19169 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:02.538055 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d43236-07d4-4ada-8657-ad39aa327136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:02.539316 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d43236-07d4-4ada-8657-ad39aa327136 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:02.539964 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d43236-07d4-4ada-8657-ad39aa327136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:02.540625 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d43236-07d4-4ada-8657-ad39aa327136 admin admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:02.542649 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d43236-07d4-4ada-8657-ad39aa327136 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:17:02.543276 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d43236-07d4-4ada-8657-ad39aa327136 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:02.546582 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d43236-07d4-4ada-8657-ad39aa327136 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:02.549511 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:02.571606 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1006/2015] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:17:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:17:02.577780 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33336b94-0298-43b3-bbaa-1350401af083 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:02.591750 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33336b94-0298-43b3-bbaa-1350401af083 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:02.612925 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f89b0-96e5-447c-ad19-8688958ad6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:02.614116 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f89b0-96e5-447c-ad19-8688958ad6aa admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:02.614402 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f89b0-96e5-447c-ad19-8688958ad6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:02.614682 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f89b0-96e5-447c-ad19-8688958ad6aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:02.616743 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6f89b0-96e5-447c-ad19-8688958ad6aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:02.622704 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6f89b0-96e5-447c-ad19-8688958ad6aa admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:02.691920 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1010/2016] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:17:02 2022] POST /identity/v3/projects => generated 381 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:17:02.698458 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d43236-07d4-4ada-8657-ad39aa327136 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:02.717072 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d43236-07d4-4ada-8657-ad39aa327136 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:02.828544 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d9f94a-9761-4155-afce-f45f1403e5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:02.829373 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9f94a-9761-4155-afce-f45f1403e5ef admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:02.829731 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9f94a-9761-4155-afce-f45f1403e5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:02.830031 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9f94a-9761-4155-afce-f45f1403e5ef admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:02.830769 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d9f94a-9761-4155-afce-f45f1403e5ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:02.832859 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d9f94a-9761-4155-afce-f45f1403e5ef admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:02.872315 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1007/2017] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:17:02 2022] POST /identity/v3/users => generated 291 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:17:02.874292 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f89b0-96e5-447c-ad19-8688958ad6aa admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:02.892624 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f89b0-96e5-447c-ad19-8688958ad6aa admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:02.897815 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8df5b5f-64d6-41b5-aeae-46d32ef020dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:02.898501 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df5b5f-64d6-41b5-aeae-46d32ef020dc admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:02.898786 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df5b5f-64d6-41b5-aeae-46d32ef020dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:02.899061 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df5b5f-64d6-41b5-aeae-46d32ef020dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:02.899906 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8df5b5f-64d6-41b5-aeae-46d32ef020dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:02.902140 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8df5b5f-64d6-41b5-aeae-46d32ef020dc admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:02.921033 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1011/2018] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:17:02.924113 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d9f94a-9761-4155-afce-f45f1403e5ef admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:02.942576 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d9f94a-9761-4155-afce-f45f1403e5ef admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:02.950512 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ccac7f-16e5-40a8-834e-9df33c70e50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:02.951424 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ccac7f-16e5-40a8-834e-9df33c70e50c admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:02.951840 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ccac7f-16e5-40a8-834e-9df33c70e50c admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:02.952213 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ccac7f-16e5-40a8-834e-9df33c70e50c admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:02.953116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ccac7f-16e5-40a8-834e-9df33c70e50c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:02.955212 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ccac7f-16e5-40a8-834e-9df33c70e50c admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:02.971484 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1008/2019] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:17:02.980376 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8df5b5f-64d6-41b5-aeae-46d32ef020dc admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:02.998905 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8df5b5f-64d6-41b5-aeae-46d32ef020dc admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:03.006969 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d7e421-1a52-4636-b713-417be49d00aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:03.008526 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7e421-1a52-4636-b713-417be49d00aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:03.008993 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7e421-1a52-4636-b713-417be49d00aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:03.009461 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7e421-1a52-4636-b713-417be49d00aa admin admin] PATH_INFO: `/v3/projects/530cb0c2252f46278332b7a12b8d8a9f/users/c91c137da7b44cba9d812a96fb5aae57/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:03.031613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:03.042732 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d7e421-1a52-4636-b713-417be49d00aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=530cb0c2252f46278332b7a12b8d8a9f, user_id=c91c137da7b44cba9d812a96fb5aae57, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:03.047069 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d7e421-1a52-4636-b713-417be49d00aa admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:03.071923 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1012/2020] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:17:02 2022] PUT /identity/v3/projects/530cb0c2252f46278332b7a12b8d8a9f/users/c91c137da7b44cba9d812a96fb5aae57/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:03.096695 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14adf98c-f3b0-4d74-9683-f17879596242 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:03.097196 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14adf98c-f3b0-4d74-9683-f17879596242 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:03.097717 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14adf98c-f3b0-4d74-9683-f17879596242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:03.145611 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-14adf98c-f3b0-4d74-9683-f17879596242 None None] MFA Rules not processed for user `c91c137da7b44cba9d812a96fb5aae57`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:03.147266 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-14adf98c-f3b0-4d74-9683-f17879596242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:03.236412 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1009/2021] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:03.241352 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d7e421-1a52-4636-b713-417be49d00aa admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:03.337433 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d7e421-1a52-4636-b713-417be49d00aa admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:03.343941 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8512a6-a9f2-457c-b6d5-f26135950343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:03.344760 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8512a6-a9f2-457c-b6d5-f26135950343 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:03.345135 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8512a6-a9f2-457c-b6d5-f26135950343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:03.345519 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8512a6-a9f2-457c-b6d5-f26135950343 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:03.346323 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8512a6-a9f2-457c-b6d5-f26135950343 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:03.349092 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8512a6-a9f2-457c-b6d5-f26135950343 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:03.387281 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1013/2022] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:17:03 2022] POST /identity/v3/projects => generated 383 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:17:03.392261 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14adf98c-f3b0-4d74-9683-f17879596242 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:03.406353 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14adf98c-f3b0-4d74-9683-f17879596242 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:03.493186 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f895f-35fb-43f9-aad3-265b7f4bd1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:03.496437 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f895f-35fb-43f9-aad3-265b7f4bd1d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:03.496883 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f895f-35fb-43f9-aad3-265b7f4bd1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:03.497304 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f895f-35fb-43f9-aad3-265b7f4bd1d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:03.498158 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404f895f-35fb-43f9-aad3-265b7f4bd1d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:03.502455 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404f895f-35fb-43f9-aad3-265b7f4bd1d8 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:03.512457 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:03.533054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1010/2023] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:17:03 2022] POST /identity/v3/users => generated 298 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:17:03.538092 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8512a6-a9f2-457c-b6d5-f26135950343 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:03.550038 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8512a6-a9f2-457c-b6d5-f26135950343 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:03.557222 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe5a97-bc37-42b0-9dee-0ad9fadaa454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:03.558197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe5a97-bc37-42b0-9dee-0ad9fadaa454 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:03.558668 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe5a97-bc37-42b0-9dee-0ad9fadaa454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:03.559103 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe5a97-bc37-42b0-9dee-0ad9fadaa454 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:03.561635 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abe5a97-bc37-42b0-9dee-0ad9fadaa454 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:03.563673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abe5a97-bc37-42b0-9dee-0ad9fadaa454 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:03.577162 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1014/2024] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:17:03.581349 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f895f-35fb-43f9-aad3-265b7f4bd1d8 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:03.602305 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f895f-35fb-43f9-aad3-265b7f4bd1d8 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:03.608165 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4cce1f-94ef-49db-a461-90c829802c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:03.609905 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4cce1f-94ef-49db-a461-90c829802c52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:03.610320 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4cce1f-94ef-49db-a461-90c829802c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:03.610710 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4cce1f-94ef-49db-a461-90c829802c52 admin admin] PATH_INFO: `/v3/projects/1521b0b409c0418bb9297fd44b2f5fb2/users/4c456ed727b2474789f591ff5349d3a3/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:03.630372 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4cce1f-94ef-49db-a461-90c829802c52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1521b0b409c0418bb9297fd44b2f5fb2, user_id=4c456ed727b2474789f591ff5349d3a3, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:03.633767 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4cce1f-94ef-49db-a461-90c829802c52 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:03.658384 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1011/2025] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:17:03 2022] PUT /identity/v3/projects/1521b0b409c0418bb9297fd44b2f5fb2/users/4c456ed727b2474789f591ff5349d3a3/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:03.659447 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe5a97-bc37-42b0-9dee-0ad9fadaa454 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:03.677785 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe5a97-bc37-42b0-9dee-0ad9fadaa454 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:03.764366 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2474dc-e6c2-4fad-880d-3014f3f55423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:03.765307 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2474dc-e6c2-4fad-880d-3014f3f55423 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:03.765687 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2474dc-e6c2-4fad-880d-3014f3f55423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:03.766071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2474dc-e6c2-4fad-880d-3014f3f55423 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:03.767473 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2474dc-e6c2-4fad-880d-3014f3f55423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:03.769866 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2474dc-e6c2-4fad-880d-3014f3f55423 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:03.783150 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1015/2026] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:17:03.788363 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4cce1f-94ef-49db-a461-90c829802c52 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:03.803795 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4cce1f-94ef-49db-a461-90c829802c52 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:03.815257 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e5ce9d-cdf9-4fa2-a8fb-8c6290c989d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:03.816352 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e5ce9d-cdf9-4fa2-a8fb-8c6290c989d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:03.816735 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e5ce9d-cdf9-4fa2-a8fb-8c6290c989d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:03.817071 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e5ce9d-cdf9-4fa2-a8fb-8c6290c989d5 admin admin] PATH_INFO: `/v3/projects/1521b0b409c0418bb9297fd44b2f5fb2/users/4c456ed727b2474789f591ff5349d3a3/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:03.819760 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e5ce9d-cdf9-4fa2-a8fb-8c6290c989d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1521b0b409c0418bb9297fd44b2f5fb2, user_id=4c456ed727b2474789f591ff5349d3a3, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:03.822572 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e5ce9d-cdf9-4fa2-a8fb-8c6290c989d5 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:03.837014 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1012/2027] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:17:03 2022] PUT /identity/v3/projects/1521b0b409c0418bb9297fd44b2f5fb2/users/4c456ed727b2474789f591ff5349d3a3/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:03.850539 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff2ae9-310d-4f31-815e-24c95839b63d None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:03.850879 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff2ae9-310d-4f31-815e-24c95839b63d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:03.851171 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff2ae9-310d-4f31-815e-24c95839b63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:03.861471 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d26ad1-44c9-4881-9ba5-9e3822cf4aa0 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:03.862826 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d26ad1-44c9-4881-9ba5-9e3822cf4aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:03.862826 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d26ad1-44c9-4881-9ba5-9e3822cf4aa0 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:03.864051 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1013/2028] 10.222.0.60 () {54 vars in 921 bytes} [Wed Jul 13 23:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:17:03.874316 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab21c3-08d7-467d-955f-f933579cbb36 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:03.874916 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab21c3-08d7-467d-955f-f933579cbb36 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:03.875676 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab21c3-08d7-467d-955f-f933579cbb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:03.885706 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-f4ff2ae9-310d-4f31-815e-24c95839b63d None None] MFA Rules not processed for user `4c456ed727b2474789f591ff5349d3a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:03.887070 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f4ff2ae9-310d-4f31-815e-24c95839b63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:03.914515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-85ab21c3-08d7-467d-955f-f933579cbb36 None None] MFA Rules not processed for user `64824ba84a024930bbc177f8a8d9a046`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:03.916890 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-85ab21c3-08d7-467d-955f-f933579cbb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:04.027403 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:04.039880 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1014/2029] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Jul 13 23:17:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:17:04.073766 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ab21c3-08d7-467d-955f-f933579cbb36 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:04.077047 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-85ab21c3-08d7-467d-955f-f933579cbb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:04.092425 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1016/2030] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:03 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:17:04.105287 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ff2ae9-310d-4f31-815e-24c95839b63d None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:04.107517 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f4ff2ae9-310d-4f31-815e-24c95839b63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:04.110130 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ab21c3-08d7-467d-955f-f933579cbb36 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:04.119375 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b655e-d45a-4b3d-bef5-6ed9caaf31cc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64824ba84a024930bbc177f8a8d9a046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:04.120668 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b655e-d45a-4b3d-bef5-6ed9caaf31cc service placement] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:04.121314 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b655e-d45a-4b3d-bef5-6ed9caaf31cc service placement] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:04.121939 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b655e-d45a-4b3d-bef5-6ed9caaf31cc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:04.140964 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4b655e-d45a-4b3d-bef5-6ed9caaf31cc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:04.142897 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4b655e-d45a-4b3d-bef5-6ed9caaf31cc service placement] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:04.158439 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1015/2031] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:17:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:17:04.222764 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ff2ae9-310d-4f31-815e-24c95839b63d None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:04.230487 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8b1d2-41f1-4eac-94ca-67567e9f40eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:04.231611 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8b1d2-41f1-4eac-94ca-67567e9f40eb service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:04.232291 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8b1d2-41f1-4eac-94ca-67567e9f40eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:04.232740 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8b1d2-41f1-4eac-94ca-67567e9f40eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:04.246918 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8b1d2-41f1-4eac-94ca-67567e9f40eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:04.248791 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8b1d2-41f1-4eac-94ca-67567e9f40eb service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:04.266236 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1017/2032] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:17:04 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:17:04.403624 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b655e-d45a-4b3d-bef5-6ed9caaf31cc service placement] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:04.415129 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b655e-d45a-4b3d-bef5-6ed9caaf31cc service placement] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:04.421555 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e024170-de36-474a-bb35-1724d7b41304 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:04.422718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e024170-de36-474a-bb35-1724d7b41304 service cinder] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:04.423275 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e024170-de36-474a-bb35-1724d7b41304 service cinder] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:04.423888 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e024170-de36-474a-bb35-1724d7b41304 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:04.510288 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e024170-de36-474a-bb35-1724d7b41304 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:04.511981 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e024170-de36-474a-bb35-1724d7b41304 service cinder] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:04.525745 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1016/2033] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:17:04 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:17:04.802563 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8b1d2-41f1-4eac-94ca-67567e9f40eb service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:04.808364 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-50b8b1d2-41f1-4eac-94ca-67567e9f40eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:04.977374 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8b1d2-41f1-4eac-94ca-67567e9f40eb service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:04.986085 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd6435c-2687-4278-a937-b346f56ebb88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:04.987430 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6435c-2687-4278-a937-b346f56ebb88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:04.987993 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6435c-2687-4278-a937-b346f56ebb88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:04.988452 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6435c-2687-4278-a937-b346f56ebb88 admin admin] PATH_INFO: `/v3/users/3a8e5e7596a44fac93dd07e9d17e7566` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:04.991020 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd6435c-2687-4278-a937-b346f56ebb88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a8e5e7596a44fac93dd07e9d17e7566)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:04.995688 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd6435c-2687-4278-a937-b346f56ebb88 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:04.996215 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e024170-de36-474a-bb35-1724d7b41304 service cinder] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:05.013425 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e024170-de36-474a-bb35-1724d7b41304 service cinder] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:05.020553 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5df26-74f6-4cea-a7ed-3f04d42eb6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:05.021956 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5df26-74f6-4cea-a7ed-3f04d42eb6e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:05.022370 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5df26-74f6-4cea-a7ed-3f04d42eb6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:05.022797 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5df26-74f6-4cea-a7ed-3f04d42eb6e4 admin admin] PATH_INFO: `/v3/users/c91c137da7b44cba9d812a96fb5aae57` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:05.024574 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c5df26-74f6-4cea-a7ed-3f04d42eb6e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c91c137da7b44cba9d812a96fb5aae57)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:05.027474 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c5df26-74f6-4cea-a7ed-3f04d42eb6e4 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:05.087487 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:INSERT=4,keystone:SELECT=91,keystone:DELETE=5 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:05.108482 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ffd6435c-2687-4278-a937-b346f56ebb88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a8e5e7596a44fac93dd07e9d17e7566'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:05.109565 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ffd6435c-2687-4278-a937-b346f56ebb88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a8e5e7596a44fac93dd07e9d17e7566'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:05.113000 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-c7c5df26-74f6-4cea-a7ed-3f04d42eb6e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c91c137da7b44cba9d812a96fb5aae57'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:05.114134 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-c7c5df26-74f6-4cea-a7ed-3f04d42eb6e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c91c137da7b44cba9d812a96fb5aae57'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:05.134587 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-c7c5df26-74f6-4cea-a7ed-3f04d42eb6e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c91c137da7b44cba9d812a96fb5aae57'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:05.135104 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ffd6435c-2687-4278-a937-b346f56ebb88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a8e5e7596a44fac93dd07e9d17e7566'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:05.156955 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ffd6435c-2687-4278-a937-b346f56ebb88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a8e5e7596a44fac93dd07e9d17e7566'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:05.163877 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-c7c5df26-74f6-4cea-a7ed-3f04d42eb6e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c91c137da7b44cba9d812a96fb5aae57'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:05.165236 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-c7c5df26-74f6-4cea-a7ed-3f04d42eb6e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c91c137da7b44cba9d812a96fb5aae57'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:05.178691 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ffd6435c-2687-4278-a937-b346f56ebb88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a8e5e7596a44fac93dd07e9d17e7566'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:05.199893 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1018/2035] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:17:04 2022] DELETE /identity/v3/users/3a8e5e7596a44fac93dd07e9d17e7566 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:05.203563 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1017/2035] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:17:04 2022] DELETE /identity/v3/users/c91c137da7b44cba9d812a96fb5aae57 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:17:05.205928 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd1912-05d5-4da0-acd5-2ac380090068 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:05.206522 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd1912-05d5-4da0-acd5-2ac380090068 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:05.206965 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd1912-05d5-4da0-acd5-2ac380090068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:05.240834 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-d0dd1912-05d5-4da0-acd5-2ac380090068 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:05.242932 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d0dd1912-05d5-4da0-acd5-2ac380090068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:05.265267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:05.421098 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1019/2036] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:05.445050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5df26-74f6-4cea-a7ed-3f04d42eb6e4 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:05.447974 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-c7c5df26-74f6-4cea-a7ed-3f04d42eb6e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:05.554653 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd1912-05d5-4da0-acd5-2ac380090068 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:05.556853 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d0dd1912-05d5-4da0-acd5-2ac380090068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:05.568833 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5df26-74f6-4cea-a7ed-3f04d42eb6e4 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:05.578231 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd1912-05d5-4da0-acd5-2ac380090068 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:05.588931 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44400206-0031-4bf0-ba8e-b3fef661a807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:05.590143 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44400206-0031-4bf0-ba8e-b3fef661a807 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:05.590689 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44400206-0031-4bf0-ba8e-b3fef661a807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:05.591217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44400206-0031-4bf0-ba8e-b3fef661a807 admin admin] PATH_INFO: `/v3/projects/6cc4805d9511488ca2759f2aef15a357` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:05.594137 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44400206-0031-4bf0-ba8e-b3fef661a807 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cc4805d9511488ca2759f2aef15a357)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:05.597505 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44400206-0031-4bf0-ba8e-b3fef661a807 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:05.674220 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68566e32-9b63-4a64-905d-c6dc7ac679e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:05.675266 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68566e32-9b63-4a64-905d-c6dc7ac679e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:05.676081 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68566e32-9b63-4a64-905d-c6dc7ac679e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:05.676838 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68566e32-9b63-4a64-905d-c6dc7ac679e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:05.677465 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-44400206-0031-4bf0-ba8e-b3fef661a807 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cc4805d9511488ca2759f2aef15a357', 'request_id': 'req-44400206-0031-4bf0-ba8e-b3fef661a807'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:05.691716 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-44400206-0031-4bf0-ba8e-b3fef661a807 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cc4805d9511488ca2759f2aef15a357', 'request_id': 'req-44400206-0031-4bf0-ba8e-b3fef661a807'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:05.694093 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68566e32-9b63-4a64-905d-c6dc7ac679e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:05.696547 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68566e32-9b63-4a64-905d-c6dc7ac679e4 service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:05.711652 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-44400206-0031-4bf0-ba8e-b3fef661a807 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cc4805d9511488ca2759f2aef15a357', 'request_id': 'req-44400206-0031-4bf0-ba8e-b3fef661a807'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:05.715237 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1018/2037] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:17:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:17:05.743815 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-44400206-0031-4bf0-ba8e-b3fef661a807 admin admin] The token cache is being invalidate because project 6cc4805d9511488ca2759f2aef15a357 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:17:05.744188 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-44400206-0031-4bf0-ba8e-b3fef661a807 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:05.745050 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-44400206-0031-4bf0-ba8e-b3fef661a807 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:05.747900 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1020/2038] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:17:05 2022] DELETE /identity/v3/projects/6cc4805d9511488ca2759f2aef15a357 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:05.753801 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68566e32-9b63-4a64-905d-c6dc7ac679e4 service neutron] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:05.772478 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68566e32-9b63-4a64-905d-c6dc7ac679e4 service neutron] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:05.876429 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72b9416-de0e-4868-8297-92013ec32d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:05.877547 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b9416-de0e-4868-8297-92013ec32d89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:05.877941 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b9416-de0e-4868-8297-92013ec32d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:05.878414 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72b9416-de0e-4868-8297-92013ec32d89 admin admin] PATH_INFO: `/v3/users/ffa766792e7448da8a64eba9e2f07aa8` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:05.880154 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72b9416-de0e-4868-8297-92013ec32d89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffa766792e7448da8a64eba9e2f07aa8)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:05.882465 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72b9416-de0e-4868-8297-92013ec32d89 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:05.891823 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:05.961384 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-b72b9416-de0e-4868-8297-92013ec32d89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffa766792e7448da8a64eba9e2f07aa8'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:05.962513 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-b72b9416-de0e-4868-8297-92013ec32d89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffa766792e7448da8a64eba9e2f07aa8'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:05.979337 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-b72b9416-de0e-4868-8297-92013ec32d89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffa766792e7448da8a64eba9e2f07aa8'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:05.990153 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-b72b9416-de0e-4868-8297-92013ec32d89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffa766792e7448da8a64eba9e2f07aa8'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:05.991066 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-b72b9416-de0e-4868-8297-92013ec32d89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffa766792e7448da8a64eba9e2f07aa8'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:06.014472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1019/2039] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:17:05 2022] DELETE /identity/v3/users/ffa766792e7448da8a64eba9e2f07aa8 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:06.134085 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44400206-0031-4bf0-ba8e-b3fef661a807 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:06.136714 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-44400206-0031-4bf0-ba8e-b3fef661a807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:06.165027 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44400206-0031-4bf0-ba8e-b3fef661a807 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:06.274353 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f58d67-c8c3-4d9d-b5ac-2b055c9093e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:06.276109 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f58d67-c8c3-4d9d-b5ac-2b055c9093e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:06.276588 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f58d67-c8c3-4d9d-b5ac-2b055c9093e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:06.277000 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f58d67-c8c3-4d9d-b5ac-2b055c9093e4 admin admin] PATH_INFO: `/v3/projects/530cb0c2252f46278332b7a12b8d8a9f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:06.278728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f58d67-c8c3-4d9d-b5ac-2b055c9093e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=530cb0c2252f46278332b7a12b8d8a9f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:06.281761 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f58d67-c8c3-4d9d-b5ac-2b055c9093e4 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:06.302660 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72b9416-de0e-4868-8297-92013ec32d89 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:06.304661 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-b72b9416-de0e-4868-8297-92013ec32d89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:06.320715 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72b9416-de0e-4868-8297-92013ec32d89 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:06.338582 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3839c180-09da-4129-a946-625374667a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:06.339906 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839c180-09da-4129-a946-625374667a8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:06.340416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839c180-09da-4129-a946-625374667a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:06.340826 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839c180-09da-4129-a946-625374667a8a admin admin] PATH_INFO: `/v3/projects/632cfd2af352489ea38e08274d28274e` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:06.342929 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3839c180-09da-4129-a946-625374667a8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=632cfd2af352489ea38e08274d28274e)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:06.345724 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3839c180-09da-4129-a946-625374667a8a admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:06.358881 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:06.382470 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-78f58d67-c8c3-4d9d-b5ac-2b055c9093e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '530cb0c2252f46278332b7a12b8d8a9f', 'request_id': 'req-78f58d67-c8c3-4d9d-b5ac-2b055c9093e4'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:06.391827 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-78f58d67-c8c3-4d9d-b5ac-2b055c9093e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '530cb0c2252f46278332b7a12b8d8a9f', 'request_id': 'req-78f58d67-c8c3-4d9d-b5ac-2b055c9093e4'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:06.404301 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-78f58d67-c8c3-4d9d-b5ac-2b055c9093e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '530cb0c2252f46278332b7a12b8d8a9f', 'request_id': 'req-78f58d67-c8c3-4d9d-b5ac-2b055c9093e4'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:06.420275 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-78f58d67-c8c3-4d9d-b5ac-2b055c9093e4 admin admin] The token cache is being invalidate because project 530cb0c2252f46278332b7a12b8d8a9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:17:06.420695 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-78f58d67-c8c3-4d9d-b5ac-2b055c9093e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:06.421570 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-78f58d67-c8c3-4d9d-b5ac-2b055c9093e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:06.423782 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1021/2040] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:17:06 2022] DELETE /identity/v3/projects/530cb0c2252f46278332b7a12b8d8a9f => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:06.428107 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f58d67-c8c3-4d9d-b5ac-2b055c9093e4 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:06.430523 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-78f58d67-c8c3-4d9d-b5ac-2b055c9093e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:06.431423 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-3839c180-09da-4129-a946-625374667a8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '632cfd2af352489ea38e08274d28274e', 'request_id': 'req-3839c180-09da-4129-a946-625374667a8a'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:06.440371 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-3839c180-09da-4129-a946-625374667a8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '632cfd2af352489ea38e08274d28274e', 'request_id': 'req-3839c180-09da-4129-a946-625374667a8a'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:06.451431 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-3839c180-09da-4129-a946-625374667a8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '632cfd2af352489ea38e08274d28274e', 'request_id': 'req-3839c180-09da-4129-a946-625374667a8a'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:06.466956 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-3839c180-09da-4129-a946-625374667a8a admin admin] The token cache is being invalidate because project 632cfd2af352489ea38e08274d28274e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:17:06.467271 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-3839c180-09da-4129-a946-625374667a8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:06.468014 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-3839c180-09da-4129-a946-625374667a8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:06.469814 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1020/2041] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:17:06 2022] DELETE /identity/v3/projects/632cfd2af352489ea38e08274d28274e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:17:06.477137 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c0d82-1d79-49d0-a483-2e15bd0488cd None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:06.477510 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c0d82-1d79-49d0-a483-2e15bd0488cd None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:06.477894 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c0d82-1d79-49d0-a483-2e15bd0488cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:06.503291 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-337c0d82-1d79-49d0-a483-2e15bd0488cd None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:06.504648 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-337c0d82-1d79-49d0-a483-2e15bd0488cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:06.557606 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f58d67-c8c3-4d9d-b5ac-2b055c9093e4 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:06.634837 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1021/2042] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:17:06.641486 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666a8c2-40c3-4f27-850b-b08af232b241 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:06.642396 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666a8c2-40c3-4f27-850b-b08af232b241 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:06.643202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666a8c2-40c3-4f27-850b-b08af232b241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:06.677401 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc195b6-fb4c-4d0e-9c44-50447c1c3e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:06.677720 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-b666a8c2-40c3-4f27-850b-b08af232b241 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:06.677906 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc195b6-fb4c-4d0e-9c44-50447c1c3e7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:06.678071 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-b666a8c2-40c3-4f27-850b-b08af232b241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:06.678477 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc195b6-fb4c-4d0e-9c44-50447c1c3e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:06.679870 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc195b6-fb4c-4d0e-9c44-50447c1c3e7b admin admin] PATH_INFO: `/v3/users/4c456ed727b2474789f591ff5349d3a3` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:06.683990 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc195b6-fb4c-4d0e-9c44-50447c1c3e7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c456ed727b2474789f591ff5349d3a3)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:06.690227 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc195b6-fb4c-4d0e-9c44-50447c1c3e7b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:06.691986 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1022/2043] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:06.698060 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b666a8c2-40c3-4f27-850b-b08af232b241 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:06.703757 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:06.712749 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b666a8c2-40c3-4f27-850b-b08af232b241 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:06.720792 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae9288-e974-4445-b824-dcf2e7ef778b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:06.721849 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae9288-e974-4445-b824-dcf2e7ef778b admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:06.722958 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae9288-e974-4445-b824-dcf2e7ef778b admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:06.723834 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae9288-e974-4445-b824-dcf2e7ef778b admin admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:06.725000 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ae9288-e974-4445-b824-dcf2e7ef778b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:17:06.725530 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ae9288-e974-4445-b824-dcf2e7ef778b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:06.727523 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ae9288-e974-4445-b824-dcf2e7ef778b admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:06.750098 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1023/2044] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:17:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:17:06.756681 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae9288-e974-4445-b824-dcf2e7ef778b admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:06.770714 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae9288-e974-4445-b824-dcf2e7ef778b admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:06.778024 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a517a8-866e-4aa8-9616-54de59e35a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:06.779018 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a517a8-866e-4aa8-9616-54de59e35a08 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:06.779678 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a517a8-866e-4aa8-9616-54de59e35a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:06.780223 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a517a8-866e-4aa8-9616-54de59e35a08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:06.781766 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a517a8-866e-4aa8-9616-54de59e35a08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:06.784076 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a517a8-866e-4aa8-9616-54de59e35a08 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:06.794549 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-afc195b6-fb4c-4d0e-9c44-50447c1c3e7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c456ed727b2474789f591ff5349d3a3'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:06.795972 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-afc195b6-fb4c-4d0e-9c44-50447c1c3e7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c456ed727b2474789f591ff5349d3a3'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:06.822936 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1024/2045] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:17:06 2022] POST /identity/v3/projects => generated 381 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:17:06.827405 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a517a8-866e-4aa8-9616-54de59e35a08 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:06.830808 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-afc195b6-fb4c-4d0e-9c44-50447c1c3e7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c456ed727b2474789f591ff5349d3a3'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:06.831387 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-71a517a8-866e-4aa8-9616-54de59e35a08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:06.845369 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:06.852377 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-afc195b6-fb4c-4d0e-9c44-50447c1c3e7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c456ed727b2474789f591ff5349d3a3'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:06.879348 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-afc195b6-fb4c-4d0e-9c44-50447c1c3e7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c456ed727b2474789f591ff5349d3a3'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:06.890871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1022/2046] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:17:06 2022] DELETE /identity/v3/users/4c456ed727b2474789f591ff5349d3a3 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:17:06.950381 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a517a8-866e-4aa8-9616-54de59e35a08 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:07.039974 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8645aa3c-0392-45cf-bf97-aec7563d4cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:07.041491 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8645aa3c-0392-45cf-bf97-aec7563d4cb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:07.041974 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8645aa3c-0392-45cf-bf97-aec7563d4cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:07.042402 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8645aa3c-0392-45cf-bf97-aec7563d4cb9 admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:07.043586 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8645aa3c-0392-45cf-bf97-aec7563d4cb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:07.046866 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8645aa3c-0392-45cf-bf97-aec7563d4cb9 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:07.081539 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1025/2047] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:17:06 2022] POST /identity/v3/users => generated 291 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:17:07.086778 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc195b6-fb4c-4d0e-9c44-50447c1c3e7b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:07.104095 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc195b6-fb4c-4d0e-9c44-50447c1c3e7b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:07.110061 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e93faac-45d3-4e4a-abf6-de0471c2329f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:07.110760 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93faac-45d3-4e4a-abf6-de0471c2329f admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:07.111262 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93faac-45d3-4e4a-abf6-de0471c2329f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:07.111457 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93faac-45d3-4e4a-abf6-de0471c2329f admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:07.112092 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e93faac-45d3-4e4a-abf6-de0471c2329f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:07.113754 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e93faac-45d3-4e4a-abf6-de0471c2329f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:07.127844 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1023/2048] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:17:07 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:17:07.131252 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8645aa3c-0392-45cf-bf97-aec7563d4cb9 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:07.142500 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8645aa3c-0392-45cf-bf97-aec7563d4cb9 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:07.147756 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec683cc-ed4f-480e-a2bd-a3f317ce13f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:07.148456 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec683cc-ed4f-480e-a2bd-a3f317ce13f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:07.148733 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec683cc-ed4f-480e-a2bd-a3f317ce13f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:07.149005 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec683cc-ed4f-480e-a2bd-a3f317ce13f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:07.149749 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec683cc-ed4f-480e-a2bd-a3f317ce13f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:07.151995 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec683cc-ed4f-480e-a2bd-a3f317ce13f3 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:07.164675 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1026/2049] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:17:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:17:07.171617 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e93faac-45d3-4e4a-abf6-de0471c2329f admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:07.189606 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e93faac-45d3-4e4a-abf6-de0471c2329f admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:07.196174 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df0371b-ce81-4a3b-abe3-a75c8b048abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:07.197917 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0371b-ce81-4a3b-abe3-a75c8b048abe admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:07.198391 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0371b-ce81-4a3b-abe3-a75c8b048abe admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:07.198791 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0371b-ce81-4a3b-abe3-a75c8b048abe admin admin] PATH_INFO: `/v3/projects/06b66fcae3ab4989b7cbaa8141e2e38f/users/09394f5b3deb4aa0ba25ceca7fc2def6/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:07.222475 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df0371b-ce81-4a3b-abe3-a75c8b048abe admin admin] RBAC: Authorizing `identity:create_grant(project_id=06b66fcae3ab4989b7cbaa8141e2e38f, user_id=09394f5b3deb4aa0ba25ceca7fc2def6, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:07.228243 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df0371b-ce81-4a3b-abe3-a75c8b048abe admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:07.252898 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1024/2050] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:17:07 2022] PUT /identity/v3/projects/06b66fcae3ab4989b7cbaa8141e2e38f/users/09394f5b3deb4aa0ba25ceca7fc2def6/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:07.268211 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f04f48f-953f-4a53-8e77-9dd3e000f225 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:07.268589 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f04f48f-953f-4a53-8e77-9dd3e000f225 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:07.268980 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f04f48f-953f-4a53-8e77-9dd3e000f225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:07.309641 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-4f04f48f-953f-4a53-8e77-9dd3e000f225 None None] MFA Rules not processed for user `09394f5b3deb4aa0ba25ceca7fc2def6`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:07.311120 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-4f04f48f-953f-4a53-8e77-9dd3e000f225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:07.373866 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:07.417895 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1027/2051] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:07.419643 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df0371b-ce81-4a3b-abe3-a75c8b048abe admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:07.542061 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df0371b-ce81-4a3b-abe3-a75c8b048abe admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:07.550720 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab5dd9-7b0c-4bd7-8591-936dc1d3ac00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:07.551936 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab5dd9-7b0c-4bd7-8591-936dc1d3ac00 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:07.552428 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab5dd9-7b0c-4bd7-8591-936dc1d3ac00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:07.552912 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab5dd9-7b0c-4bd7-8591-936dc1d3ac00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:07.554481 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baab5dd9-7b0c-4bd7-8591-936dc1d3ac00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:07.559414 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baab5dd9-7b0c-4bd7-8591-936dc1d3ac00 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:07.612053 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1025/2052] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:17:07 2022] POST /identity/v3/projects => generated 385 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:17:07.613496 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f04f48f-953f-4a53-8e77-9dd3e000f225 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:07.641487 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f04f48f-953f-4a53-8e77-9dd3e000f225 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:07.742804 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86375601-fdd2-4f84-9937-e8d29a57a78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:07.743515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86375601-fdd2-4f84-9937-e8d29a57a78b admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:07.744369 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86375601-fdd2-4f84-9937-e8d29a57a78b admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:07.744645 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86375601-fdd2-4f84-9937-e8d29a57a78b admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:07.745398 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86375601-fdd2-4f84-9937-e8d29a57a78b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:07.747437 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86375601-fdd2-4f84-9937-e8d29a57a78b admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:07.778108 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1028/2053] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:17:07 2022] POST /identity/v3/users => generated 299 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:17:07.780139 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab5dd9-7b0c-4bd7-8591-936dc1d3ac00 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:07.795275 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab5dd9-7b0c-4bd7-8591-936dc1d3ac00 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:07.811476 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c0574f-4f52-48d3-8265-9020fcb7d056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:07.812891 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0574f-4f52-48d3-8265-9020fcb7d056 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:07.813526 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0574f-4f52-48d3-8265-9020fcb7d056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:07.814141 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0574f-4f52-48d3-8265-9020fcb7d056 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:07.815859 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c0574f-4f52-48d3-8265-9020fcb7d056 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:07.819731 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c0574f-4f52-48d3-8265-9020fcb7d056 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:07.835377 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86375601-fdd2-4f84-9937-e8d29a57a78b admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:07.838209 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-86375601-fdd2-4f84-9937-e8d29a57a78b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:07.849102 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1026/2054] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:17:07 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:17:07.851103 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c0574f-4f52-48d3-8265-9020fcb7d056 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:07.866357 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86375601-fdd2-4f84-9937-e8d29a57a78b admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:07.872063 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c0574f-4f52-48d3-8265-9020fcb7d056 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:07.878941 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a199104-d78f-4e18-9233-027f6a03ff76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:07.879851 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a199104-d78f-4e18-9233-027f6a03ff76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:07.880152 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a199104-d78f-4e18-9233-027f6a03ff76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:07.880411 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a199104-d78f-4e18-9233-027f6a03ff76 admin admin] PATH_INFO: `/v3/projects/60841def21b84c41b97dac3ce85fc79c/users/58926a36972b46c883688962e7f45d56/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:07.892326 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170e4fa6-49d2-44d5-b1a5-e5663128c2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:07.893259 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e4fa6-49d2-44d5-b1a5-e5663128c2a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:07.893569 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e4fa6-49d2-44d5-b1a5-e5663128c2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:07.893894 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e4fa6-49d2-44d5-b1a5-e5663128c2a8 admin admin] PATH_INFO: `/v3/projects/1521b0b409c0418bb9297fd44b2f5fb2` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:07.895964 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170e4fa6-49d2-44d5-b1a5-e5663128c2a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1521b0b409c0418bb9297fd44b2f5fb2)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:07.901406 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170e4fa6-49d2-44d5-b1a5-e5663128c2a8 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:07.904087 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a199104-d78f-4e18-9233-027f6a03ff76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60841def21b84c41b97dac3ce85fc79c, user_id=58926a36972b46c883688962e7f45d56, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:07.907050 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a199104-d78f-4e18-9233-027f6a03ff76 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:07.930565 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1027/2055] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:17:07 2022] PUT /identity/v3/projects/60841def21b84c41b97dac3ce85fc79c/users/58926a36972b46c883688962e7f45d56/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:07.934262 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a199104-d78f-4e18-9233-027f6a03ff76 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:08.028426 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a199104-d78f-4e18-9233-027f6a03ff76 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:08.032759 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-170e4fa6-49d2-44d5-b1a5-e5663128c2a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1521b0b409c0418bb9297fd44b2f5fb2', 'request_id': 'req-170e4fa6-49d2-44d5-b1a5-e5663128c2a8'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:08.039955 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:08.058852 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-170e4fa6-49d2-44d5-b1a5-e5663128c2a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1521b0b409c0418bb9297fd44b2f5fb2', 'request_id': 'req-170e4fa6-49d2-44d5-b1a5-e5663128c2a8'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:08.075210 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-170e4fa6-49d2-44d5-b1a5-e5663128c2a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1521b0b409c0418bb9297fd44b2f5fb2', 'request_id': 'req-170e4fa6-49d2-44d5-b1a5-e5663128c2a8'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:08.100409 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-170e4fa6-49d2-44d5-b1a5-e5663128c2a8 admin admin] The token cache is being invalidate because project 1521b0b409c0418bb9297fd44b2f5fb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:17:08.100967 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-170e4fa6-49d2-44d5-b1a5-e5663128c2a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:08.102254 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-170e4fa6-49d2-44d5-b1a5-e5663128c2a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:08.106473 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1029/2056] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:17:07 2022] DELETE /identity/v3/projects/1521b0b409c0418bb9297fd44b2f5fb2 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:08.118775 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84225bd-7c21-44d7-8ab7-c104675e2b5b None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:08.119795 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84225bd-7c21-44d7-8ab7-c104675e2b5b None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:08.120447 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84225bd-7c21-44d7-8ab7-c104675e2b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:08.130897 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0457fb-146c-4106-9887-622b08d54ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:08.131976 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0457fb-146c-4106-9887-622b08d54ece admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:08.132709 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0457fb-146c-4106-9887-622b08d54ece admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:08.133179 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0457fb-146c-4106-9887-622b08d54ece admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:08.135213 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0457fb-146c-4106-9887-622b08d54ece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:08.137231 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0457fb-146c-4106-9887-622b08d54ece admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:08.156161 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1028/2057] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:17:07 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:17:08.157578 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-d84225bd-7c21-44d7-8ab7-c104675e2b5b None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:08.159152 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-d84225bd-7c21-44d7-8ab7-c104675e2b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:08.161415 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0457fb-146c-4106-9887-622b08d54ece admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:08.164190 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7b0457fb-146c-4106-9887-622b08d54ece admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:08.168614 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1030/2058] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:08.178293 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811dfa50-d916-438f-b7ae-0dc84a922564 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:08.179314 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811dfa50-d916-438f-b7ae-0dc84a922564 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:08.180220 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811dfa50-d916-438f-b7ae-0dc84a922564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:08.183731 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0457fb-146c-4106-9887-622b08d54ece admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:08.191225 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd285c12-ae72-4cb8-88ff-01d5fbb66345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:08.192778 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd285c12-ae72-4cb8-88ff-01d5fbb66345 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:08.193246 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd285c12-ae72-4cb8-88ff-01d5fbb66345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:08.193712 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd285c12-ae72-4cb8-88ff-01d5fbb66345 admin admin] PATH_INFO: `/v3/projects/60841def21b84c41b97dac3ce85fc79c/users/58926a36972b46c883688962e7f45d56/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:08.197273 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd285c12-ae72-4cb8-88ff-01d5fbb66345 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60841def21b84c41b97dac3ce85fc79c, user_id=58926a36972b46c883688962e7f45d56, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:08.202405 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd285c12-ae72-4cb8-88ff-01d5fbb66345 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:08.213083 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-811dfa50-d916-438f-b7ae-0dc84a922564 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:08.215302 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-811dfa50-d916-438f-b7ae-0dc84a922564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:08.221163 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1029/2059] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:17:08 2022] PUT /identity/v3/projects/60841def21b84c41b97dac3ce85fc79c/users/58926a36972b46c883688962e7f45d56/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:08.239715 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb87af-432e-45b7-81e6-af3079b0c1dd None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:08.240186 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb87af-432e-45b7-81e6-af3079b0c1dd None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:08.240645 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb87af-432e-45b7-81e6-af3079b0c1dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:08.287509 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-3edb87af-432e-45b7-81e6-af3079b0c1dd None None] MFA Rules not processed for user `58926a36972b46c883688962e7f45d56`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:08.289703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3edb87af-432e-45b7-81e6-af3079b0c1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:08.292255 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:08.317559 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1031/2060] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:08.319241 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811dfa50-d916-438f-b7ae-0dc84a922564 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:08.331330 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811dfa50-d916-438f-b7ae-0dc84a922564 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:08.338665 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e362d62-b2dc-4d43-9c1b-3851ce9e58a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:08.339413 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e362d62-b2dc-4d43-9c1b-3851ce9e58a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:08.340291 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e362d62-b2dc-4d43-9c1b-3851ce9e58a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:08.340575 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e362d62-b2dc-4d43-9c1b-3851ce9e58a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:08.341400 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e362d62-b2dc-4d43-9c1b-3851ce9e58a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:17:08.341715 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e362d62-b2dc-4d43-9c1b-3851ce9e58a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:08.343679 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e362d62-b2dc-4d43-9c1b-3851ce9e58a0 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:08.370883 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1032/2061] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:17:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:17:08.376367 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e362d62-b2dc-4d43-9c1b-3851ce9e58a0 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:08.387682 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e362d62-b2dc-4d43-9c1b-3851ce9e58a0 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:08.392801 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c26a7c-2fd0-4508-bc5c-db0fa8301a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:08.393569 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c26a7c-2fd0-4508-bc5c-db0fa8301a84 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:08.393964 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c26a7c-2fd0-4508-bc5c-db0fa8301a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:08.394454 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c26a7c-2fd0-4508-bc5c-db0fa8301a84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:08.395188 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c26a7c-2fd0-4508-bc5c-db0fa8301a84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:08.397259 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c26a7c-2fd0-4508-bc5c-db0fa8301a84 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:08.423416 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1030/2062] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:08 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:08.436718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1033/2063] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:17:08 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:17:08.439514 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb87af-432e-45b7-81e6-af3079b0c1dd None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:08.442351 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c26a7c-2fd0-4508-bc5c-db0fa8301a84 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:08.444271 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-32c26a7c-2fd0-4508-bc5c-db0fa8301a84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:08.462272 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb87af-432e-45b7-81e6-af3079b0c1dd None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:08.517678 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c26a7c-2fd0-4508-bc5c-db0fa8301a84 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:08.527397 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52fbf37-9276-4d5a-ab5d-f8da9ec8ef8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:08.528104 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52fbf37-9276-4d5a-ab5d-f8da9ec8ef8f service cinder] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:08.528383 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52fbf37-9276-4d5a-ab5d-f8da9ec8ef8f service cinder] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:08.528655 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52fbf37-9276-4d5a-ab5d-f8da9ec8ef8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:08.532084 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-d52fbf37-9276-4d5a-ab5d-f8da9ec8ef8f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:08.544686 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e9b51-35f4-4613-9d49-54a0e5ee2759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:08.545607 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e9b51-35f4-4613-9d49-54a0e5ee2759 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:08.545607 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e9b51-35f4-4613-9d49-54a0e5ee2759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:08.546015 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e9b51-35f4-4613-9d49-54a0e5ee2759 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:08.546500 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27e9b51-35f4-4613-9d49-54a0e5ee2759 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:08.548546 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27e9b51-35f4-4613-9d49-54a0e5ee2759 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:08.573197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1031/2064] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:17:08 2022] POST /identity/v3/users => generated 290 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:17:08.577098 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e9b51-35f4-4613-9d49-54a0e5ee2759 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:08.597653 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e9b51-35f4-4613-9d49-54a0e5ee2759 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:08.617657 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85dfd9e-9ee9-4bed-b980-6ddfdf96712d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:08.619511 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85dfd9e-9ee9-4bed-b980-6ddfdf96712d admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:08.620863 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85dfd9e-9ee9-4bed-b980-6ddfdf96712d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:08.623494 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85dfd9e-9ee9-4bed-b980-6ddfdf96712d admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:08.625897 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85dfd9e-9ee9-4bed-b980-6ddfdf96712d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:08.629700 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85dfd9e-9ee9-4bed-b980-6ddfdf96712d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:08.637098 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:08.639842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52fbf37-9276-4d5a-ab5d-f8da9ec8ef8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:08.641668 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52fbf37-9276-4d5a-ab5d-f8da9ec8ef8f service cinder] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:08.657633 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1032/2065] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:17:08 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:17:08.660213 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1034/2066] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:17:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:17:08.662659 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85dfd9e-9ee9-4bed-b980-6ddfdf96712d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:08.681927 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85dfd9e-9ee9-4bed-b980-6ddfdf96712d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:08.691826 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb0eb4f-0f09-4b46-b913-bb62892b081b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:08.692945 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb0eb4f-0f09-4b46-b913-bb62892b081b admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:08.693671 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb0eb4f-0f09-4b46-b913-bb62892b081b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:08.694167 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb0eb4f-0f09-4b46-b913-bb62892b081b admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:08.695950 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb0eb4f-0f09-4b46-b913-bb62892b081b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:08.700220 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb0eb4f-0f09-4b46-b913-bb62892b081b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:08.725091 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1033/2067] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:17:08 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:17:08.732414 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52fbf37-9276-4d5a-ab5d-f8da9ec8ef8f service cinder] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:08.766536 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52fbf37-9276-4d5a-ab5d-f8da9ec8ef8f service cinder] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:08.780708 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dbe9a7-161a-4c48-8ad8-b4d79b387f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:08.782292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dbe9a7-161a-4c48-8ad8-b4d79b387f58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:08.783384 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dbe9a7-161a-4c48-8ad8-b4d79b387f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:08.788041 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dbe9a7-161a-4c48-8ad8-b4d79b387f58 admin admin] PATH_INFO: `/v3/projects/d631f17810f94dcc91dfcb28a4b17c9a/users/c3871ceed47e4fcea07c5f879c8462cb/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:08.809090 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dbe9a7-161a-4c48-8ad8-b4d79b387f58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d631f17810f94dcc91dfcb28a4b17c9a, user_id=c3871ceed47e4fcea07c5f879c8462cb, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:08.812521 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dbe9a7-161a-4c48-8ad8-b4d79b387f58 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:08.835639 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1035/2068] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:17:08 2022] PUT /identity/v3/projects/d631f17810f94dcc91dfcb28a4b17c9a/users/c3871ceed47e4fcea07c5f879c8462cb/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:08.839442 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448fb251-1287-4c13-9fd4-ce2b86b5335e None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:08.840085 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448fb251-1287-4c13-9fd4-ce2b86b5335e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:08.840654 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448fb251-1287-4c13-9fd4-ce2b86b5335e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:08.865123 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-448fb251-1287-4c13-9fd4-ce2b86b5335e None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:08.866676 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-448fb251-1287-4c13-9fd4-ce2b86b5335e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:08.956657 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1034/2069] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:17:08.966362 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dbe9a7-161a-4c48-8ad8-b4d79b387f58 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:08.968283 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-b0dbe9a7-161a-4c48-8ad8-b4d79b387f58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:09.025341 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:09.045455 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dbe9a7-161a-4c48-8ad8-b4d79b387f58 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:09.054084 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e8738-5455-4931-8b29-95a0c74150fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:09.055382 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e8738-5455-4931-8b29-95a0c74150fb service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:09.056062 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e8738-5455-4931-8b29-95a0c74150fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:09.057334 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e8738-5455-4931-8b29-95a0c74150fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:09.076708 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1e8738-5455-4931-8b29-95a0c74150fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:09.078723 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1e8738-5455-4931-8b29-95a0c74150fb service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:09.095683 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1036/2070] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:17:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:17:12.218728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448fb251-1287-4c13-9fd4-ce2b86b5335e None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:12.219569 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-448fb251-1287-4c13-9fd4-ce2b86b5335e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:12.250797 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448fb251-1287-4c13-9fd4-ce2b86b5335e None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:12.260805 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086b9cb-510a-4ca0-b892-ada812202cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:12.261749 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086b9cb-510a-4ca0-b892-ada812202cb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:12.262036 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086b9cb-510a-4ca0-b892-ada812202cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:12.262313 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086b9cb-510a-4ca0-b892-ada812202cb8 admin admin] PATH_INFO: `/v3/users/7ade2669af504e91936edb12c5d919a6` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:12.264114 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3086b9cb-510a-4ca0-b892-ada812202cb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ade2669af504e91936edb12c5d919a6)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:12.266358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3086b9cb-510a-4ca0-b892-ada812202cb8 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:12.375001 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3086b9cb-510a-4ca0-b892-ada812202cb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ade2669af504e91936edb12c5d919a6'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:12.376386 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3086b9cb-510a-4ca0-b892-ada812202cb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ade2669af504e91936edb12c5d919a6'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:12.398704 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3086b9cb-510a-4ca0-b892-ada812202cb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ade2669af504e91936edb12c5d919a6'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:12.410551 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3086b9cb-510a-4ca0-b892-ada812202cb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ade2669af504e91936edb12c5d919a6'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:12.424824 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3086b9cb-510a-4ca0-b892-ada812202cb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ade2669af504e91936edb12c5d919a6'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:12.442959 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1035/2071] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:17:12 2022] DELETE /identity/v3/users/7ade2669af504e91936edb12c5d919a6 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:12.943361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e8738-5455-4931-8b29-95a0c74150fb service neutron] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:12.974357 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e8738-5455-4931-8b29-95a0c74150fb service neutron] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:13.135865 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beccdb4d-93dc-4a16-8121-34814267a96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:13.138151 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beccdb4d-93dc-4a16-8121-34814267a96a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:13.139166 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beccdb4d-93dc-4a16-8121-34814267a96a admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:13.140246 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beccdb4d-93dc-4a16-8121-34814267a96a admin admin] PATH_INFO: `/v3/projects/5c873a6e9bdd4b74bcea03fb45317291` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:13.142165 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beccdb4d-93dc-4a16-8121-34814267a96a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c873a6e9bdd4b74bcea03fb45317291)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:13.146382 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beccdb4d-93dc-4a16-8121-34814267a96a admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:13.312943 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-beccdb4d-93dc-4a16-8121-34814267a96a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c873a6e9bdd4b74bcea03fb45317291', 'request_id': 'req-beccdb4d-93dc-4a16-8121-34814267a96a'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:13.352117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-beccdb4d-93dc-4a16-8121-34814267a96a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c873a6e9bdd4b74bcea03fb45317291', 'request_id': 'req-beccdb4d-93dc-4a16-8121-34814267a96a'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:13.383065 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-beccdb4d-93dc-4a16-8121-34814267a96a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c873a6e9bdd4b74bcea03fb45317291', 'request_id': 'req-beccdb4d-93dc-4a16-8121-34814267a96a'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:13.416457 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-beccdb4d-93dc-4a16-8121-34814267a96a admin admin] The token cache is being invalidate because project 5c873a6e9bdd4b74bcea03fb45317291 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:17:13.416929 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-beccdb4d-93dc-4a16-8121-34814267a96a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:13.419625 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-beccdb4d-93dc-4a16-8121-34814267a96a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:13.436491 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1037/2072] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:17:12 2022] DELETE /identity/v3/projects/5c873a6e9bdd4b74bcea03fb45317291 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:17:13.440331 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9331bb07-0023-4382-ad79-84fde00091f8 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:13.441879 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9331bb07-0023-4382-ad79-84fde00091f8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:13.442652 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9331bb07-0023-4382-ad79-84fde00091f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:13.474712 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-9331bb07-0023-4382-ad79-84fde00091f8 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:13.477155 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9331bb07-0023-4382-ad79-84fde00091f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:13.555128 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:13.566896 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1036/2073] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:13.577208 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169cbef0-e715-4ca3-bc70-54c0d9e70728 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:13.577482 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169cbef0-e715-4ca3-bc70-54c0d9e70728 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:13.577861 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169cbef0-e715-4ca3-bc70-54c0d9e70728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:13.613165 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-169cbef0-e715-4ca3-bc70-54c0d9e70728 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:13.614240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-169cbef0-e715-4ca3-bc70-54c0d9e70728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:13.627998 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1038/2074] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:13.631451 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9331bb07-0023-4382-ad79-84fde00091f8 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:13.652927 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9331bb07-0023-4382-ad79-84fde00091f8 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:13.661583 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc9aed4-1b7c-4e86-a913-13bc4ecc61cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:13.662774 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9aed4-1b7c-4e86-a913-13bc4ecc61cd admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:13.663248 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9aed4-1b7c-4e86-a913-13bc4ecc61cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:13.664512 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9aed4-1b7c-4e86-a913-13bc4ecc61cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:13.665262 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc9aed4-1b7c-4e86-a913-13bc4ecc61cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:17:13.666325 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc9aed4-1b7c-4e86-a913-13bc4ecc61cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:13.669733 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc9aed4-1b7c-4e86-a913-13bc4ecc61cd admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:13.694483 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1037/2075] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:17:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:17:13.699622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169cbef0-e715-4ca3-bc70-54c0d9e70728 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:13.714971 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169cbef0-e715-4ca3-bc70-54c0d9e70728 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:13.720870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03731553-f602-4e74-8bea-d3373f45a2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:13.721688 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03731553-f602-4e74-8bea-d3373f45a2da admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:13.721970 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03731553-f602-4e74-8bea-d3373f45a2da admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:13.722417 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03731553-f602-4e74-8bea-d3373f45a2da admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:13.723149 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03731553-f602-4e74-8bea-d3373f45a2da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:13.725468 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03731553-f602-4e74-8bea-d3373f45a2da admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:13.761855 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1039/2076] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:17:13 2022] POST /identity/v3/projects => generated 373 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:17:13.765632 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc9aed4-1b7c-4e86-a913-13bc4ecc61cd admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:13.778202 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc9aed4-1b7c-4e86-a913-13bc4ecc61cd admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:13.881146 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d40b4c3-f076-480b-918c-4a7a58f364c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:13.882573 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40b4c3-f076-480b-918c-4a7a58f364c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:13.883157 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40b4c3-f076-480b-918c-4a7a58f364c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:13.883772 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40b4c3-f076-480b-918c-4a7a58f364c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:13.885683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d40b4c3-f076-480b-918c-4a7a58f364c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:13.889717 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d40b4c3-f076-480b-918c-4a7a58f364c0 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:13.937734 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1038/2077] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:17:13 2022] POST /identity/v3/users => generated 287 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:17:13.939005 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03731553-f602-4e74-8bea-d3373f45a2da admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:13.951520 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03731553-f602-4e74-8bea-d3373f45a2da admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:13.959754 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334fb18-e7fe-418c-99cb-00f3f0154fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:13.960449 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334fb18-e7fe-418c-99cb-00f3f0154fa7 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:13.960733 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334fb18-e7fe-418c-99cb-00f3f0154fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:13.961009 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9334fb18-e7fe-418c-99cb-00f3f0154fa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:13.961993 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9334fb18-e7fe-418c-99cb-00f3f0154fa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:13.964691 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9334fb18-e7fe-418c-99cb-00f3f0154fa7 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:13.979963 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1040/2078] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:17:13 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:17:13.986814 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d40b4c3-f076-480b-918c-4a7a58f364c0 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:13.999112 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d40b4c3-f076-480b-918c-4a7a58f364c0 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:14.006262 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a076025-b8fc-4dc0-83c7-5b54e6fdb238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:14.007113 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a076025-b8fc-4dc0-83c7-5b54e6fdb238 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:14.007470 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a076025-b8fc-4dc0-83c7-5b54e6fdb238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:14.007887 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a076025-b8fc-4dc0-83c7-5b54e6fdb238 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:14.008771 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a076025-b8fc-4dc0-83c7-5b54e6fdb238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:14.011239 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a076025-b8fc-4dc0-83c7-5b54e6fdb238 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:14.031745 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1039/2079] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:17:13 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:17:14.033270 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334fb18-e7fe-418c-99cb-00f3f0154fa7 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:14.045430 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9334fb18-e7fe-418c-99cb-00f3f0154fa7 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:14.051367 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736ed31-0cdf-41fd-91b1-31f094fb1e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:14.052544 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736ed31-0cdf-41fd-91b1-31f094fb1e87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:14.053046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736ed31-0cdf-41fd-91b1-31f094fb1e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:14.053752 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736ed31-0cdf-41fd-91b1-31f094fb1e87 admin admin] PATH_INFO: `/v3/projects/8b784daabc424ce39047311e91dddbea/users/c9db90073b784b32aea583440370aa0e/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:14.078436 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d736ed31-0cdf-41fd-91b1-31f094fb1e87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b784daabc424ce39047311e91dddbea, user_id=c9db90073b784b32aea583440370aa0e, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:14.081771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d736ed31-0cdf-41fd-91b1-31f094fb1e87 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:14.115363 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1041/2080] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:17:14 2022] PUT /identity/v3/projects/8b784daabc424ce39047311e91dddbea/users/c9db90073b784b32aea583440370aa0e/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:14.117086 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c26a06-b3ed-404b-9db9-d9a3893b58ae None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:14.117380 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c26a06-b3ed-404b-9db9-d9a3893b58ae None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:14.117797 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c26a06-b3ed-404b-9db9-d9a3893b58ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:14.166189 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-62c26a06-b3ed-404b-9db9-d9a3893b58ae None None] MFA Rules not processed for user `c9db90073b784b32aea583440370aa0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:14.167869 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-62c26a06-b3ed-404b-9db9-d9a3893b58ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:14.276471 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1040/2081] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:14.287758 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736ed31-0cdf-41fd-91b1-31f094fb1e87 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:14.289645 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-d736ed31-0cdf-41fd-91b1-31f094fb1e87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:14.310116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:14.459456 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736ed31-0cdf-41fd-91b1-31f094fb1e87 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:14.485644 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1909aca0-61ca-4916-96de-5648240fd3a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:14.487092 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1909aca0-61ca-4916-96de-5648240fd3a6 service nova] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:14.495661 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1909aca0-61ca-4916-96de-5648240fd3a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:14.495661 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1909aca0-61ca-4916-96de-5648240fd3a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:14.514905 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1909aca0-61ca-4916-96de-5648240fd3a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:14.518718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1909aca0-61ca-4916-96de-5648240fd3a6 service nova] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:14.554788 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1042/2082] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:17:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:17:14.558064 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c26a06-b3ed-404b-9db9-d9a3893b58ae None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:14.559846 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-62c26a06-b3ed-404b-9db9-d9a3893b58ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:14.661219 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c26a06-b3ed-404b-9db9-d9a3893b58ae None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:14.666550 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dba228-3413-4228-928c-e2d6da8e6bbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:14.667380 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dba228-3413-4228-928c-e2d6da8e6bbb service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:14.667802 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dba228-3413-4228-928c-e2d6da8e6bbb service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:14.668390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dba228-3413-4228-928c-e2d6da8e6bbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:14.671209 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e9dba228-3413-4228-928c-e2d6da8e6bbb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:14.683916 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:14.803916 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dba228-3413-4228-928c-e2d6da8e6bbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:14.805490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dba228-3413-4228-928c-e2d6da8e6bbb service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:14.833936 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1041/2083] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:17:14 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:17:14.883972 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1909aca0-61ca-4916-96de-5648240fd3a6 service nova] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:15.029036 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1909aca0-61ca-4916-96de-5648240fd3a6 service nova] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:15.039659 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1f023-2c38-497c-aadd-7daf48493248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:15.040721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1f023-2c38-497c-aadd-7daf48493248 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:15.041099 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1f023-2c38-497c-aadd-7daf48493248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:15.041475 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1f023-2c38-497c-aadd-7daf48493248 admin admin] PATH_INFO: `/v3/users/c9db90073b784b32aea583440370aa0e` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:15.043174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd1f023-2c38-497c-aadd-7daf48493248 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9db90073b784b32aea583440370aa0e)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:15.045951 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd1f023-2c38-497c-aadd-7daf48493248 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:15.125145 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-3fd1f023-2c38-497c-aadd-7daf48493248 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9db90073b784b32aea583440370aa0e'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:15.125805 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-3fd1f023-2c38-497c-aadd-7daf48493248 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9db90073b784b32aea583440370aa0e'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:15.142585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-3fd1f023-2c38-497c-aadd-7daf48493248 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9db90073b784b32aea583440370aa0e'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:15.153155 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-3fd1f023-2c38-497c-aadd-7daf48493248 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9db90073b784b32aea583440370aa0e'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:15.154325 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-3fd1f023-2c38-497c-aadd-7daf48493248 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9db90073b784b32aea583440370aa0e'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:15.176927 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1043/2084] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:17:14 2022] DELETE /identity/v3/users/c9db90073b784b32aea583440370aa0e => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:15.183672 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a929c7-564e-41e4-8ff5-1feb62c2e00e None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:15.183969 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a929c7-564e-41e4-8ff5-1feb62c2e00e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:15.184252 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a929c7-564e-41e4-8ff5-1feb62c2e00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:15.209751 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-49a929c7-564e-41e4-8ff5-1feb62c2e00e None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:15.211229 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-49a929c7-564e-41e4-8ff5-1feb62c2e00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:15.341719 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1042/2085] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:15.348391 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1f023-2c38-497c-aadd-7daf48493248 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:15.350796 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-3fd1f023-2c38-497c-aadd-7daf48493248 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:15.395560 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:15.454638 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1f023-2c38-497c-aadd-7daf48493248 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:15.462319 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69441a2-924d-4a56-a323-7a8762178e25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:15.463503 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69441a2-924d-4a56-a323-7a8762178e25 service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:15.464309 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69441a2-924d-4a56-a323-7a8762178e25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:15.464822 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69441a2-924d-4a56-a323-7a8762178e25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:15.479779 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69441a2-924d-4a56-a323-7a8762178e25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:15.482046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69441a2-924d-4a56-a323-7a8762178e25 service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:15.495562 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1044/2086] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:17:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:17:16.166514 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a929c7-564e-41e4-8ff5-1feb62c2e00e None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:16.169465 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-49a929c7-564e-41e4-8ff5-1feb62c2e00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:16.195656 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a929c7-564e-41e4-8ff5-1feb62c2e00e None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:16.204888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94598fb-2296-4933-a59e-0731fd441c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:16.206571 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94598fb-2296-4933-a59e-0731fd441c20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:16.207071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94598fb-2296-4933-a59e-0731fd441c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:16.207667 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94598fb-2296-4933-a59e-0731fd441c20 admin admin] PATH_INFO: `/v3/projects/8b784daabc424ce39047311e91dddbea` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:16.210195 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94598fb-2296-4933-a59e-0731fd441c20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b784daabc424ce39047311e91dddbea)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:16.214112 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94598fb-2296-4933-a59e-0731fd441c20 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:16.321672 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c94598fb-2296-4933-a59e-0731fd441c20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b784daabc424ce39047311e91dddbea', 'request_id': 'req-c94598fb-2296-4933-a59e-0731fd441c20'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:16.333417 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c94598fb-2296-4933-a59e-0731fd441c20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b784daabc424ce39047311e91dddbea', 'request_id': 'req-c94598fb-2296-4933-a59e-0731fd441c20'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:16.352429 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c94598fb-2296-4933-a59e-0731fd441c20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b784daabc424ce39047311e91dddbea', 'request_id': 'req-c94598fb-2296-4933-a59e-0731fd441c20'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:16.382102 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c94598fb-2296-4933-a59e-0731fd441c20 admin admin] The token cache is being invalidate because project 8b784daabc424ce39047311e91dddbea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:17:16.382605 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c94598fb-2296-4933-a59e-0731fd441c20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:16.383425 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c94598fb-2296-4933-a59e-0731fd441c20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:16.385686 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1043/2087] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:17:16 2022] DELETE /identity/v3/projects/8b784daabc424ce39047311e91dddbea => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:16.391019 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e748894-8bec-48cd-9232-7a9ac8979827 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:16.391950 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e748894-8bec-48cd-9232-7a9ac8979827 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:16.392461 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e748894-8bec-48cd-9232-7a9ac8979827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:16.412344 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-0e748894-8bec-48cd-9232-7a9ac8979827 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:16.414172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-0e748894-8bec-48cd-9232-7a9ac8979827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:16.530480 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1045/2088] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:16.533050 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1cf9e-6fbc-4d1e-b6a1-f275f09d42a5 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:16.534457 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1cf9e-6fbc-4d1e-b6a1-f275f09d42a5 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:16.534951 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1cf9e-6fbc-4d1e-b6a1-f275f09d42a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:16.585051 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-38c1cf9e-6fbc-4d1e-b6a1-f275f09d42a5 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:16.589072 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-38c1cf9e-6fbc-4d1e-b6a1-f275f09d42a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:16.599667 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1044/2089] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:16.607631 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e748894-8bec-48cd-9232-7a9ac8979827 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:16.637528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e748894-8bec-48cd-9232-7a9ac8979827 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:16.646727 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade0667-94c3-4998-9433-81d9ebaf5e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:16.647847 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade0667-94c3-4998-9433-81d9ebaf5e5c admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:16.648154 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade0667-94c3-4998-9433-81d9ebaf5e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:16.648438 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade0667-94c3-4998-9433-81d9ebaf5e5c admin admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:16.654189 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ade0667-94c3-4998-9433-81d9ebaf5e5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:17:16.654445 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ade0667-94c3-4998-9433-81d9ebaf5e5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:16.657353 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ade0667-94c3-4998-9433-81d9ebaf5e5c admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:16.674373 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1046/2090] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:17:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:17:16.681682 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1cf9e-6fbc-4d1e-b6a1-f275f09d42a5 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:16.699826 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1cf9e-6fbc-4d1e-b6a1-f275f09d42a5 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:16.717888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ce13ee-bf46-4a06-b3fe-dfc2162b35a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:16.720466 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce13ee-bf46-4a06-b3fe-dfc2162b35a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:16.720979 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce13ee-bf46-4a06-b3fe-dfc2162b35a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:16.721472 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce13ee-bf46-4a06-b3fe-dfc2162b35a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:16.726695 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ce13ee-bf46-4a06-b3fe-dfc2162b35a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:16.740045 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ce13ee-bf46-4a06-b3fe-dfc2162b35a7 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:16.781805 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:16.803696 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1045/2091] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:17:16 2022] POST /identity/v3/projects => generated 369 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:17:16.809940 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade0667-94c3-4998-9433-81d9ebaf5e5c admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:16.861794 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade0667-94c3-4998-9433-81d9ebaf5e5c admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:17.049775 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0ea3d-dfaa-482c-a144-b050e77a0c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:17.050749 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0ea3d-dfaa-482c-a144-b050e77a0c92 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:17.051498 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0ea3d-dfaa-482c-a144-b050e77a0c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:17.052078 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0ea3d-dfaa-482c-a144-b050e77a0c92 admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:17.055064 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb0ea3d-dfaa-482c-a144-b050e77a0c92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:17.059585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb0ea3d-dfaa-482c-a144-b050e77a0c92 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:17.102523 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d743a-b315-470e-887d-68cea9fa08bf None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:17.103037 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d743a-b315-470e-887d-68cea9fa08bf None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:17.104324 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d743a-b315-470e-887d-68cea9fa08bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:17.110705 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1047/2092] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:17:16 2022] POST /identity/v3/users => generated 285 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:17:17.112838 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0ea3d-dfaa-482c-a144-b050e77a0c92 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:17.137549 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0ea3d-dfaa-482c-a144-b050e77a0c92 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:17.148315 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dcf935-8c4e-4b6d-9637-a7e55dcb12f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:17.150449 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dcf935-8c4e-4b6d-9637-a7e55dcb12f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:17.152511 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dcf935-8c4e-4b6d-9637-a7e55dcb12f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:17.153382 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dcf935-8c4e-4b6d-9637-a7e55dcb12f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:17.159519 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dcf935-8c4e-4b6d-9637-a7e55dcb12f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:17.164447 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dcf935-8c4e-4b6d-9637-a7e55dcb12f3 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:17.177767 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c47d743a-b315-470e-887d-68cea9fa08bf None None] MFA Rules not processed for user `c3871ceed47e4fcea07c5f879c8462cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:17.179867 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c47d743a-b315-470e-887d-68cea9fa08bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:17.192835 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1048/2093] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:17:17 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:17:17.199669 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dcf935-8c4e-4b6d-9637-a7e55dcb12f3 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:17.219728 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dcf935-8c4e-4b6d-9637-a7e55dcb12f3 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:17.228710 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50b32fa-20df-4b43-ba3c-36bd2b6a5a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:17.229639 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b32fa-20df-4b43-ba3c-36bd2b6a5a25 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:17.230053 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b32fa-20df-4b43-ba3c-36bd2b6a5a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:17.230616 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b32fa-20df-4b43-ba3c-36bd2b6a5a25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:17.231498 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50b32fa-20df-4b43-ba3c-36bd2b6a5a25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:17.233494 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50b32fa-20df-4b43-ba3c-36bd2b6a5a25 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:17.251614 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1049/2094] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:17:17 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:17:17.256349 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50b32fa-20df-4b43-ba3c-36bd2b6a5a25 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:17.278072 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50b32fa-20df-4b43-ba3c-36bd2b6a5a25 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:17.289428 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275e5dc-0cdd-484c-b6d1-a22c54a54da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:17.291657 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275e5dc-0cdd-484c-b6d1-a22c54a54da9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:17.291889 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275e5dc-0cdd-484c-b6d1-a22c54a54da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:17.292377 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275e5dc-0cdd-484c-b6d1-a22c54a54da9 admin admin] PATH_INFO: `/v3/projects/6530ddc5a813461082b0d82a02895e0b/users/3e44c0e7e04b4477919382847d8399c0/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:17.317398 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:17.336669 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275e5dc-0cdd-484c-b6d1-a22c54a54da9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6530ddc5a813461082b0d82a02895e0b, user_id=3e44c0e7e04b4477919382847d8399c0, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:17.338689 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1046/2095] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:17 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:17:17.339856 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275e5dc-0cdd-484c-b6d1-a22c54a54da9 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:17.344762 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d743a-b315-470e-887d-68cea9fa08bf None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:17.346350 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c47d743a-b315-470e-887d-68cea9fa08bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:17.368229 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1050/2096] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:17:17 2022] PUT /identity/v3/projects/6530ddc5a813461082b0d82a02895e0b/users/3e44c0e7e04b4477919382847d8399c0/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:17.369084 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d743a-b315-470e-887d-68cea9fa08bf None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:17.391365 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659df463-ccce-4653-bcc6-2154839ead77 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:17.392618 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659df463-ccce-4653-bcc6-2154839ead77 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:17.393473 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659df463-ccce-4653-bcc6-2154839ead77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:17.455309 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-659df463-ccce-4653-bcc6-2154839ead77 None None] MFA Rules not processed for user `3e44c0e7e04b4477919382847d8399c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:17.458244 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-659df463-ccce-4653-bcc6-2154839ead77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:17.459072 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcbe50a-94c3-4387-8910-53dc89d7ffd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:17.459960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcbe50a-94c3-4387-8910-53dc89d7ffd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:17.460250 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcbe50a-94c3-4387-8910-53dc89d7ffd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:17.460532 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcbe50a-94c3-4387-8910-53dc89d7ffd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:17.461284 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcbe50a-94c3-4387-8910-53dc89d7ffd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:17.463319 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcbe50a-94c3-4387-8910-53dc89d7ffd6 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:17.506359 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1047/2097] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:17:17 2022] POST /identity/v3/projects => generated 379 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:17:17.513031 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcbe50a-94c3-4387-8910-53dc89d7ffd6 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:17.630427 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1051/2098] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:17 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:17:17.636541 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcbe50a-94c3-4387-8910-53dc89d7ffd6 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:17.637998 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659df463-ccce-4653-bcc6-2154839ead77 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:17.669378 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44f536-a826-4f2b-a0d4-7b4c05494a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:17.671201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659df463-ccce-4653-bcc6-2154839ead77 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:17.680352 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44f536-a826-4f2b-a0d4-7b4c05494a37 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:17.681537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44f536-a826-4f2b-a0d4-7b4c05494a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:17.682044 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44f536-a826-4f2b-a0d4-7b4c05494a37 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:17.683416 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b44f536-a826-4f2b-a0d4-7b4c05494a37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:17.690065 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f48a8-ebc7-485d-a92f-c064414d89ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:17.690791 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f48a8-ebc7-485d-a92f-c064414d89ec admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:17.691216 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f48a8-ebc7-485d-a92f-c064414d89ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:17.691499 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f48a8-ebc7-485d-a92f-c064414d89ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:17.692274 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b44f536-a826-4f2b-a0d4-7b4c05494a37 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:17.692432 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6f48a8-ebc7-485d-a92f-c064414d89ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:17.694459 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6f48a8-ebc7-485d-a92f-c064414d89ec admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:17.733849 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1052/2099] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:17:17 2022] POST /identity/v3/projects => generated 367 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:17:17.739494 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f48a8-ebc7-485d-a92f-c064414d89ec admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:17.741350 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1048/2100] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:17:17 2022] POST /identity/v3/users => generated 296 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:17:17.743028 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44f536-a826-4f2b-a0d4-7b4c05494a37 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:17.756099 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:17.766630 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b44f536-a826-4f2b-a0d4-7b4c05494a37 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:17.863018 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f48a8-ebc7-485d-a92f-c064414d89ec admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:17.866057 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b7a768-2bca-48b0-852d-a603d2709838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:17.867309 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7a768-2bca-48b0-852d-a603d2709838 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:17.867767 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7a768-2bca-48b0-852d-a603d2709838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:17.868212 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7a768-2bca-48b0-852d-a603d2709838 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:17.869617 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b7a768-2bca-48b0-852d-a603d2709838 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:17.871406 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b7a768-2bca-48b0-852d-a603d2709838 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:17.872701 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df4da70-84dc-4821-9317-c6cea3f3ea72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:17.874641 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4da70-84dc-4821-9317-c6cea3f3ea72 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:17.875149 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4da70-84dc-4821-9317-c6cea3f3ea72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:17.875676 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4da70-84dc-4821-9317-c6cea3f3ea72 admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:17.877015 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df4da70-84dc-4821-9317-c6cea3f3ea72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:17.881845 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df4da70-84dc-4821-9317-c6cea3f3ea72 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:17.888476 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1049/2101] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:17:17 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:17:17.893510 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b7a768-2bca-48b0-852d-a603d2709838 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:17.913626 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b7a768-2bca-48b0-852d-a603d2709838 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:17.921410 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1053/2102] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:17:17 2022] POST /identity/v3/users => generated 290 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:17:17.925535 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d0a41-1e9c-4f87-8bed-9bb41db38ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:17.927209 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d0a41-1e9c-4f87-8bed-9bb41db38ef4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:17.927717 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d0a41-1e9c-4f87-8bed-9bb41db38ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:17.928146 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d0a41-1e9c-4f87-8bed-9bb41db38ef4 admin admin] PATH_INFO: `/v3/projects/49f42da3393441458d857e5f8d642df0/users/2453702c7e11464d8aefea3614549ae9/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:17.928720 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df4da70-84dc-4821-9317-c6cea3f3ea72 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:17.948011 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df4da70-84dc-4821-9317-c6cea3f3ea72 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:17.948579 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d0a41-1e9c-4f87-8bed-9bb41db38ef4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49f42da3393441458d857e5f8d642df0, user_id=2453702c7e11464d8aefea3614549ae9, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:17.951306 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d0a41-1e9c-4f87-8bed-9bb41db38ef4 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:17.956529 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e00be-c916-48f2-a419-5e54262bb2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:17.957718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e00be-c916-48f2-a419-5e54262bb2c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:17.958226 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e00be-c916-48f2-a419-5e54262bb2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:17.958735 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e00be-c916-48f2-a419-5e54262bb2c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:17.960208 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2e00be-c916-48f2-a419-5e54262bb2c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:17.963481 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2e00be-c916-48f2-a419-5e54262bb2c2 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:17.980536 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1050/2103] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:17:17 2022] PUT /identity/v3/projects/49f42da3393441458d857e5f8d642df0/users/2453702c7e11464d8aefea3614549ae9/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:17:17.982759 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d0a41-1e9c-4f87-8bed-9bb41db38ef4 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:17.985367 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1054/2104] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:17:17 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:17:17.991762 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e00be-c916-48f2-a419-5e54262bb2c2 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:18.003612 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e00be-c916-48f2-a419-5e54262bb2c2 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:18.051075 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:18.089767 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec0d035-4898-4f65-b94c-ab952eb54de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:18.090694 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0d035-4898-4f65-b94c-ab952eb54de4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:18.090976 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0d035-4898-4f65-b94c-ab952eb54de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:18.091253 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0d035-4898-4f65-b94c-ab952eb54de4 admin admin] PATH_INFO: `/v3/projects/a85913a6ccc24692a25dcdcf0f91de5a/users/3c20428c59634f1799179302aef3e6e4/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:18.109061 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d0a41-1e9c-4f87-8bed-9bb41db38ef4 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:18.117450 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b875ee-8f3a-46d9-970e-738373b320f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:18.118264 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b875ee-8f3a-46d9-970e-738373b320f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:18.118615 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b875ee-8f3a-46d9-970e-738373b320f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:18.118905 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b875ee-8f3a-46d9-970e-738373b320f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:18.119728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b875ee-8f3a-46d9-970e-738373b320f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:18.119871 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec0d035-4898-4f65-b94c-ab952eb54de4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a85913a6ccc24692a25dcdcf0f91de5a, user_id=3c20428c59634f1799179302aef3e6e4, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:18.121468 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b875ee-8f3a-46d9-970e-738373b320f5 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:18.125856 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec0d035-4898-4f65-b94c-ab952eb54de4 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:18.139643 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1051/2105] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:17:17 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:17:18.141026 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b875ee-8f3a-46d9-970e-738373b320f5 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:18.156849 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b875ee-8f3a-46d9-970e-738373b320f5 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:18.162167 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1055/2106] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:17:17 2022] PUT /identity/v3/projects/a85913a6ccc24692a25dcdcf0f91de5a/users/3c20428c59634f1799179302aef3e6e4/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:17:18.169799 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec0d035-4898-4f65-b94c-ab952eb54de4 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:18.262677 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b11c3-bb17-401f-949b-a276a332b064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:18.263780 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b11c3-bb17-401f-949b-a276a332b064 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:18.264002 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b11c3-bb17-401f-949b-a276a332b064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:18.264265 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b11c3-bb17-401f-949b-a276a332b064 admin admin] PATH_INFO: `/v3/projects/49f42da3393441458d857e5f8d642df0/users/2453702c7e11464d8aefea3614549ae9/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:18.266566 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7b11c3-bb17-401f-949b-a276a332b064 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49f42da3393441458d857e5f8d642df0, user_id=2453702c7e11464d8aefea3614549ae9, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:18.269934 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7b11c3-bb17-401f-949b-a276a332b064 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:18.286940 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1052/2107] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:17:18 2022] PUT /identity/v3/projects/49f42da3393441458d857e5f8d642df0/users/2453702c7e11464d8aefea3614549ae9/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:18.357012 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec0d035-4898-4f65-b94c-ab952eb54de4 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:18.539980 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c03667b-d598-48ed-bfcf-7c4a3596d0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:18.544264 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03667b-d598-48ed-bfcf-7c4a3596d0b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:18.544706 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03667b-d598-48ed-bfcf-7c4a3596d0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:18.547467 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03667b-d598-48ed-bfcf-7c4a3596d0b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:18.549074 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c03667b-d598-48ed-bfcf-7c4a3596d0b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:18.561444 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c03667b-d598-48ed-bfcf-7c4a3596d0b2 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:18.653684 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1056/2108] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 487 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:17:18.660432 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b11c3-bb17-401f-949b-a276a332b064 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:18.676094 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b11c3-bb17-401f-949b-a276a332b064 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:18.687912 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c899651-01ef-417b-b08c-7e361f0e1038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:18.689475 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c899651-01ef-417b-b08c-7e361f0e1038 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:18.690096 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c899651-01ef-417b-b08c-7e361f0e1038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:18.690744 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c899651-01ef-417b-b08c-7e361f0e1038 admin admin] PATH_INFO: `/v3/projects/a85913a6ccc24692a25dcdcf0f91de5a/users/3c20428c59634f1799179302aef3e6e4/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:18.693957 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c899651-01ef-417b-b08c-7e361f0e1038 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a85913a6ccc24692a25dcdcf0f91de5a, user_id=3c20428c59634f1799179302aef3e6e4, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:18.701050 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c899651-01ef-417b-b08c-7e361f0e1038 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:18.708518 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:18.772096 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1053/2109] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:17:18 2022] PUT /identity/v3/projects/a85913a6ccc24692a25dcdcf0f91de5a/users/3c20428c59634f1799179302aef3e6e4/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:18.772485 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2bde91-c6ff-465a-abd6-41048ca0543c None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:18.772986 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2bde91-c6ff-465a-abd6-41048ca0543c None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:18.773475 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2bde91-c6ff-465a-abd6-41048ca0543c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:18.870630 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-1e2bde91-c6ff-465a-abd6-41048ca0543c None None] MFA Rules not processed for user `3c20428c59634f1799179302aef3e6e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:18.872532 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-1e2bde91-c6ff-465a-abd6-41048ca0543c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:18.995571 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:19.051070 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1057/2110] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:18 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:19.070389 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c899651-01ef-417b-b08c-7e361f0e1038 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:19.074335 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1c899651-01ef-417b-b08c-7e361f0e1038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:19.194054 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c899651-01ef-417b-b08c-7e361f0e1038 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:19.200043 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab04243f-b540-4b4f-962b-3fd2942256ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:19.200848 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04243f-b540-4b4f-962b-3fd2942256ae service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:19.201207 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04243f-b540-4b4f-962b-3fd2942256ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:19.201560 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04243f-b540-4b4f-962b-3fd2942256ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:19.211761 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab04243f-b540-4b4f-962b-3fd2942256ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:19.213392 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab04243f-b540-4b4f-962b-3fd2942256ae service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:19.224755 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1054/2111] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:17:19 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:17:21.103819 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2bde91-c6ff-465a-abd6-41048ca0543c None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:21.106335 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-1e2bde91-c6ff-465a-abd6-41048ca0543c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:21.360847 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2bde91-c6ff-465a-abd6-41048ca0543c None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:21.394123 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d2bcf-2fbb-43d1-bafe-1a8ffae817c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:21.395497 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d2bcf-2fbb-43d1-bafe-1a8ffae817c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:21.397059 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d2bcf-2fbb-43d1-bafe-1a8ffae817c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:21.398765 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d2bcf-2fbb-43d1-bafe-1a8ffae817c5 admin admin] PATH_INFO: `/v3/users/09394f5b3deb4aa0ba25ceca7fc2def6` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:21.401706 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d2bcf-2fbb-43d1-bafe-1a8ffae817c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09394f5b3deb4aa0ba25ceca7fc2def6)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:21.409646 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d2bcf-2fbb-43d1-bafe-1a8ffae817c5 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:21.551872 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-613d2bcf-2fbb-43d1-bafe-1a8ffae817c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09394f5b3deb4aa0ba25ceca7fc2def6'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:21.560358 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-613d2bcf-2fbb-43d1-bafe-1a8ffae817c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09394f5b3deb4aa0ba25ceca7fc2def6'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:21.607175 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-613d2bcf-2fbb-43d1-bafe-1a8ffae817c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09394f5b3deb4aa0ba25ceca7fc2def6'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:21.632497 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-613d2bcf-2fbb-43d1-bafe-1a8ffae817c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09394f5b3deb4aa0ba25ceca7fc2def6'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:21.641679 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-613d2bcf-2fbb-43d1-bafe-1a8ffae817c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09394f5b3deb4aa0ba25ceca7fc2def6'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:21.695607 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1058/2112] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:17:21 2022] DELETE /identity/v3/users/09394f5b3deb4aa0ba25ceca7fc2def6 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:21.699402 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c2bda3-9abf-4627-aae8-4b58bb066cc5 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:21.700441 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c2bda3-9abf-4627-aae8-4b58bb066cc5 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:21.700441 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c2bda3-9abf-4627-aae8-4b58bb066cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:21.776761 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-e8c2bda3-9abf-4627-aae8-4b58bb066cc5 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:21.778977 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e8c2bda3-9abf-4627-aae8-4b58bb066cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:22.006251 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1055/2113] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:22.080408 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d2bcf-2fbb-43d1-bafe-1a8ffae817c5 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:22.084546 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-613d2bcf-2fbb-43d1-bafe-1a8ffae817c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:22.345256 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d2bcf-2fbb-43d1-bafe-1a8ffae817c5 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:22.354302 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ec496-b50f-4452-9553-11436ea5e51c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:22.354998 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ec496-b50f-4452-9553-11436ea5e51c service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:22.355284 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ec496-b50f-4452-9553-11436ea5e51c service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:22.356444 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ec496-b50f-4452-9553-11436ea5e51c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:22.420602 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ec496-b50f-4452-9553-11436ea5e51c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:22.423976 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ec496-b50f-4452-9553-11436ea5e51c service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:22.454446 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1059/2114] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:17:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:17:22.636167 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c2bda3-9abf-4627-aae8-4b58bb066cc5 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:22.638590 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e8c2bda3-9abf-4627-aae8-4b58bb066cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:22.659878 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c2bda3-9abf-4627-aae8-4b58bb066cc5 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:22.667914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22f89a-2f56-4ee6-9d03-625fc002d8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:22.669481 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22f89a-2f56-4ee6-9d03-625fc002d8fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:22.670056 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22f89a-2f56-4ee6-9d03-625fc002d8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:22.670789 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22f89a-2f56-4ee6-9d03-625fc002d8fc admin admin] PATH_INFO: `/v3/users/3e44c0e7e04b4477919382847d8399c0` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:22.672857 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22f89a-2f56-4ee6-9d03-625fc002d8fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e44c0e7e04b4477919382847d8399c0)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:22.675615 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22f89a-2f56-4ee6-9d03-625fc002d8fc admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:22.865616 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7f22f89a-2f56-4ee6-9d03-625fc002d8fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e44c0e7e04b4477919382847d8399c0'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:22.869029 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7f22f89a-2f56-4ee6-9d03-625fc002d8fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e44c0e7e04b4477919382847d8399c0'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:22.952613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7f22f89a-2f56-4ee6-9d03-625fc002d8fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e44c0e7e04b4477919382847d8399c0'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:22.970896 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7f22f89a-2f56-4ee6-9d03-625fc002d8fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e44c0e7e04b4477919382847d8399c0'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:22.989906 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7f22f89a-2f56-4ee6-9d03-625fc002d8fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e44c0e7e04b4477919382847d8399c0'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:23.009047 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1056/2115] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:17:22 2022] DELETE /identity/v3/users/3e44c0e7e04b4477919382847d8399c0 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:23.013967 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabf8d4-53ac-418f-9851-5b97a4df9102 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:23.014505 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabf8d4-53ac-418f-9851-5b97a4df9102 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:23.015397 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabf8d4-53ac-418f-9851-5b97a4df9102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:23.038954 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-9eabf8d4-53ac-418f-9851-5b97a4df9102 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:23.042248 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-9eabf8d4-53ac-418f-9851-5b97a4df9102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:23.054339 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:23.117568 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22f89a-2f56-4ee6-9d03-625fc002d8fc admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:23.123003 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7f22f89a-2f56-4ee6-9d03-625fc002d8fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:23.197007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1060/2116] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:23.201006 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:23.209475 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabf8d4-53ac-418f-9851-5b97a4df9102 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:23.211608 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-9eabf8d4-53ac-418f-9851-5b97a4df9102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:23.327809 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabf8d4-53ac-418f-9851-5b97a4df9102 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:23.335185 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b54bd75-45fa-44dc-be7f-8bacc46ffe0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:23.336383 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54bd75-45fa-44dc-be7f-8bacc46ffe0d service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:23.337259 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54bd75-45fa-44dc-be7f-8bacc46ffe0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:23.338562 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54bd75-45fa-44dc-be7f-8bacc46ffe0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:23.348806 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22f89a-2f56-4ee6-9d03-625fc002d8fc admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:23.385089 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f2a56-cfe9-440b-844c-df73649dcc79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:23.386405 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f2a56-cfe9-440b-844c-df73649dcc79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:23.388015 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f2a56-cfe9-440b-844c-df73649dcc79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:23.388570 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f2a56-cfe9-440b-844c-df73649dcc79 admin admin] PATH_INFO: `/v3/projects/06b66fcae3ab4989b7cbaa8141e2e38f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:23.390831 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b54bd75-45fa-44dc-be7f-8bacc46ffe0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:23.395034 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5f2a56-cfe9-440b-844c-df73649dcc79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06b66fcae3ab4989b7cbaa8141e2e38f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:23.397484 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5f2a56-cfe9-440b-844c-df73649dcc79 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:23.398399 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b54bd75-45fa-44dc-be7f-8bacc46ffe0d service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:23.454006 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1061/2117] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:17:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:17:23.524848 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8c5f2a56-cfe9-440b-844c-df73649dcc79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06b66fcae3ab4989b7cbaa8141e2e38f', 'request_id': 'req-8c5f2a56-cfe9-440b-844c-df73649dcc79'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:23.532835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8c5f2a56-cfe9-440b-844c-df73649dcc79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06b66fcae3ab4989b7cbaa8141e2e38f', 'request_id': 'req-8c5f2a56-cfe9-440b-844c-df73649dcc79'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:23.557626 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8c5f2a56-cfe9-440b-844c-df73649dcc79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06b66fcae3ab4989b7cbaa8141e2e38f', 'request_id': 'req-8c5f2a56-cfe9-440b-844c-df73649dcc79'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:23.571848 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8c5f2a56-cfe9-440b-844c-df73649dcc79 admin admin] The token cache is being invalidate because project 06b66fcae3ab4989b7cbaa8141e2e38f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:17:23.572210 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8c5f2a56-cfe9-440b-844c-df73649dcc79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:23.573071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8c5f2a56-cfe9-440b-844c-df73649dcc79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:23.575016 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1057/2118] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:17:23 2022] DELETE /identity/v3/projects/06b66fcae3ab4989b7cbaa8141e2e38f => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:23.580664 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b54bd75-45fa-44dc-be7f-8bacc46ffe0d service neutron] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:23.608821 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b54bd75-45fa-44dc-be7f-8bacc46ffe0d service neutron] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:23.768083 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d31990a-20f1-4529-a906-696a82655366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:23.769902 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31990a-20f1-4529-a906-696a82655366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:23.770388 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31990a-20f1-4529-a906-696a82655366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:23.770885 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31990a-20f1-4529-a906-696a82655366 admin admin] PATH_INFO: `/v3/users/58926a36972b46c883688962e7f45d56` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:23.775492 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d31990a-20f1-4529-a906-696a82655366 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58926a36972b46c883688962e7f45d56)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:23.780466 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d31990a-20f1-4529-a906-696a82655366 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:23.939042 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-3d31990a-20f1-4529-a906-696a82655366 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58926a36972b46c883688962e7f45d56'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:23.940490 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-3d31990a-20f1-4529-a906-696a82655366 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58926a36972b46c883688962e7f45d56'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:23.973098 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:23.984395 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-3d31990a-20f1-4529-a906-696a82655366 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58926a36972b46c883688962e7f45d56'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:24.019663 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-3d31990a-20f1-4529-a906-696a82655366 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58926a36972b46c883688962e7f45d56'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:24.021372 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-3d31990a-20f1-4529-a906-696a82655366 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58926a36972b46c883688962e7f45d56'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:24.098309 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1062/2119] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:17:23 2022] DELETE /identity/v3/users/58926a36972b46c883688962e7f45d56 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:17:24.274013 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f2a56-cfe9-440b-844c-df73649dcc79 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:24.277348 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8c5f2a56-cfe9-440b-844c-df73649dcc79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:24.327342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f2a56-cfe9-440b-844c-df73649dcc79 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:24.530842 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61400369-6cd6-4ee5-a5af-683a24423911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:24.531771 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61400369-6cd6-4ee5-a5af-683a24423911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:24.532056 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61400369-6cd6-4ee5-a5af-683a24423911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:24.532329 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61400369-6cd6-4ee5-a5af-683a24423911 admin admin] PATH_INFO: `/v3/projects/6530ddc5a813461082b0d82a02895e0b` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:24.533883 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61400369-6cd6-4ee5-a5af-683a24423911 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6530ddc5a813461082b0d82a02895e0b)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:24.549700 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61400369-6cd6-4ee5-a5af-683a24423911 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:24.667029 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:24.683335 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-61400369-6cd6-4ee5-a5af-683a24423911 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6530ddc5a813461082b0d82a02895e0b', 'request_id': 'req-61400369-6cd6-4ee5-a5af-683a24423911'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:24.694359 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-61400369-6cd6-4ee5-a5af-683a24423911 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6530ddc5a813461082b0d82a02895e0b', 'request_id': 'req-61400369-6cd6-4ee5-a5af-683a24423911'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:24.707884 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-61400369-6cd6-4ee5-a5af-683a24423911 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6530ddc5a813461082b0d82a02895e0b', 'request_id': 'req-61400369-6cd6-4ee5-a5af-683a24423911'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:24.740295 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-61400369-6cd6-4ee5-a5af-683a24423911 admin admin] The token cache is being invalidate because project 6530ddc5a813461082b0d82a02895e0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:17:24.740905 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-61400369-6cd6-4ee5-a5af-683a24423911 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:24.742718 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-61400369-6cd6-4ee5-a5af-683a24423911 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:24.746216 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1058/2120] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:17:24 2022] DELETE /identity/v3/projects/6530ddc5a813461082b0d82a02895e0b => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:24.752117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d31990a-20f1-4529-a906-696a82655366 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:24.758608 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-3d31990a-20f1-4529-a906-696a82655366 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:24.776545 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d31990a-20f1-4529-a906-696a82655366 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:24.903630 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e376420e-b4ed-4d43-8e50-fb23290c788c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:24.907197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61400369-6cd6-4ee5-a5af-683a24423911 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:24.907753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376420e-b4ed-4d43-8e50-fb23290c788c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:24.908556 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376420e-b4ed-4d43-8e50-fb23290c788c admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:24.909118 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-61400369-6cd6-4ee5-a5af-683a24423911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:24.909631 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376420e-b4ed-4d43-8e50-fb23290c788c admin admin] PATH_INFO: `/v3/users/3c20428c59634f1799179302aef3e6e4` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:24.913574 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e376420e-b4ed-4d43-8e50-fb23290c788c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c20428c59634f1799179302aef3e6e4)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:24.920007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e376420e-b4ed-4d43-8e50-fb23290c788c admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:24.943785 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61400369-6cd6-4ee5-a5af-683a24423911 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:24.951622 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab6931-eeaa-4d92-8244-dd0f1217114b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:24.953128 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab6931-eeaa-4d92-8244-dd0f1217114b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:24.953803 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab6931-eeaa-4d92-8244-dd0f1217114b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:24.954177 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab6931-eeaa-4d92-8244-dd0f1217114b admin admin] PATH_INFO: `/v3/projects/60841def21b84c41b97dac3ce85fc79c` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:24.957865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fab6931-eeaa-4d92-8244-dd0f1217114b admin admin] RBAC: Authorizing `identity:delete_project(project_id=60841def21b84c41b97dac3ce85fc79c)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:24.960445 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fab6931-eeaa-4d92-8244-dd0f1217114b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:25.066307 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e376420e-b4ed-4d43-8e50-fb23290c788c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c20428c59634f1799179302aef3e6e4'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:25.067424 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e376420e-b4ed-4d43-8e50-fb23290c788c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c20428c59634f1799179302aef3e6e4'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:25.083461 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6fab6931-eeaa-4d92-8244-dd0f1217114b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60841def21b84c41b97dac3ce85fc79c', 'request_id': 'req-6fab6931-eeaa-4d92-8244-dd0f1217114b'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:25.099184 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e376420e-b4ed-4d43-8e50-fb23290c788c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c20428c59634f1799179302aef3e6e4'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:25.121193 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6fab6931-eeaa-4d92-8244-dd0f1217114b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60841def21b84c41b97dac3ce85fc79c', 'request_id': 'req-6fab6931-eeaa-4d92-8244-dd0f1217114b'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:25.122369 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e376420e-b4ed-4d43-8e50-fb23290c788c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c20428c59634f1799179302aef3e6e4'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:25.123252 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e376420e-b4ed-4d43-8e50-fb23290c788c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c20428c59634f1799179302aef3e6e4'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:25.146602 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1063/2121] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:17:24 2022] DELETE /identity/v3/users/3c20428c59634f1799179302aef3e6e4 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:25.147758 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6fab6931-eeaa-4d92-8244-dd0f1217114b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60841def21b84c41b97dac3ce85fc79c', 'request_id': 'req-6fab6931-eeaa-4d92-8244-dd0f1217114b'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:25.168376 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6fab6931-eeaa-4d92-8244-dd0f1217114b admin admin] The token cache is being invalidate because project 60841def21b84c41b97dac3ce85fc79c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:17:25.169146 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6fab6931-eeaa-4d92-8244-dd0f1217114b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:25.171945 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-6fab6931-eeaa-4d92-8244-dd0f1217114b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:25.174348 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1059/2122] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:17:24 2022] DELETE /identity/v3/projects/60841def21b84c41b97dac3ce85fc79c => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:25.186635 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d303b-c972-4344-8587-51fc974fc73f None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:25.187054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d303b-c972-4344-8587-51fc974fc73f None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:25.187577 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d303b-c972-4344-8587-51fc974fc73f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:25.221340 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-3c2d303b-c972-4344-8587-51fc974fc73f None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:25.224610 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-3c2d303b-c972-4344-8587-51fc974fc73f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:25.332899 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:25.375683 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1064/2123] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:25.390222 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ca54d-ce07-4c74-bb40-475f55da8bab None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:25.390779 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ca54d-ce07-4c74-bb40-475f55da8bab None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:25.391315 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ca54d-ce07-4c74-bb40-475f55da8bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:25.424718 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-824ca54d-ce07-4c74-bb40-475f55da8bab None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:25.427266 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-824ca54d-ce07-4c74-bb40-475f55da8bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:25.452527 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1060/2124] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:17:25.454746 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d303b-c972-4344-8587-51fc974fc73f None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:25.467875 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d303b-c972-4344-8587-51fc974fc73f None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:25.473669 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94617f51-452a-423a-a927-dc7e8b1acd1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:25.474563 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94617f51-452a-423a-a927-dc7e8b1acd1e admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:25.474945 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94617f51-452a-423a-a927-dc7e8b1acd1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:25.475324 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94617f51-452a-423a-a927-dc7e8b1acd1e admin admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:25.476341 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94617f51-452a-423a-a927-dc7e8b1acd1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:17:25.478014 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94617f51-452a-423a-a927-dc7e8b1acd1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:25.479948 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94617f51-452a-423a-a927-dc7e8b1acd1e admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:25.503060 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1065/2125] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:17:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:17:25.508330 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ca54d-ce07-4c74-bb40-475f55da8bab None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:25.530607 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ca54d-ce07-4c74-bb40-475f55da8bab None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:25.541414 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c890cb4-bbb2-4169-80fa-cf3849f6ff72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:25.542989 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c890cb4-bbb2-4169-80fa-cf3849f6ff72 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:25.543804 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c890cb4-bbb2-4169-80fa-cf3849f6ff72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:25.544507 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c890cb4-bbb2-4169-80fa-cf3849f6ff72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:25.547292 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c890cb4-bbb2-4169-80fa-cf3849f6ff72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:25.551744 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c890cb4-bbb2-4169-80fa-cf3849f6ff72 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:25.599395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1061/2126] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:17:25 2022] POST /identity/v3/projects => generated 375 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:17:25.606034 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94617f51-452a-423a-a927-dc7e8b1acd1e admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:25.618739 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94617f51-452a-423a-a927-dc7e8b1acd1e admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:25.800309 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076b9d1f-6279-414c-9b1d-ec4f67290746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:25.804548 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b9d1f-6279-414c-9b1d-ec4f67290746 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:25.807754 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b9d1f-6279-414c-9b1d-ec4f67290746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:25.808592 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b9d1f-6279-414c-9b1d-ec4f67290746 admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:25.811447 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076b9d1f-6279-414c-9b1d-ec4f67290746 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:25.818218 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076b9d1f-6279-414c-9b1d-ec4f67290746 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:25.889275 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1066/2127] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:17:25 2022] POST /identity/v3/users => generated 288 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:17:25.890012 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c890cb4-bbb2-4169-80fa-cf3849f6ff72 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:25.912434 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c890cb4-bbb2-4169-80fa-cf3849f6ff72 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:25.922402 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b6f78d-870d-4945-8f6b-0e225eb27d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:25.923375 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b6f78d-870d-4945-8f6b-0e225eb27d3d admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:25.923908 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b6f78d-870d-4945-8f6b-0e225eb27d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:25.924486 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b6f78d-870d-4945-8f6b-0e225eb27d3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:25.925857 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b6f78d-870d-4945-8f6b-0e225eb27d3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:25.929773 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b6f78d-870d-4945-8f6b-0e225eb27d3d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:25.962868 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076b9d1f-6279-414c-9b1d-ec4f67290746 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:25.972228 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1062/2128] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:17:25.985174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076b9d1f-6279-414c-9b1d-ec4f67290746 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:25.994900 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e19e7f-66ab-44da-b4b1-03fa0fc91bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:25.996064 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e19e7f-66ab-44da-b4b1-03fa0fc91bbc admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:25.996527 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e19e7f-66ab-44da-b4b1-03fa0fc91bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:25.997004 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e19e7f-66ab-44da-b4b1-03fa0fc91bbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:25.998439 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e19e7f-66ab-44da-b4b1-03fa0fc91bbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:26.001721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e19e7f-66ab-44da-b4b1-03fa0fc91bbc admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:26.027470 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1067/2129] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:17:26.033255 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b6f78d-870d-4945-8f6b-0e225eb27d3d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:26.055122 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b6f78d-870d-4945-8f6b-0e225eb27d3d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:26.065871 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e19e7f-66ab-44da-b4b1-03fa0fc91bbc admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:26.069676 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-69e19e7f-66ab-44da-b4b1-03fa0fc91bbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:26.077495 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200c21d5-ea3d-467d-82cb-d97643ea6d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:26.078572 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c21d5-ea3d-467d-82cb-d97643ea6d31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:26.078937 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c21d5-ea3d-467d-82cb-d97643ea6d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:26.079334 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c21d5-ea3d-467d-82cb-d97643ea6d31 admin admin] PATH_INFO: `/v3/projects/68ad7a3acf6d4636a67341c059d66568/users/138ec380578e47599b5a78031a412071/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:26.098712 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e19e7f-66ab-44da-b4b1-03fa0fc91bbc admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:26.106436 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200c21d5-ea3d-467d-82cb-d97643ea6d31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68ad7a3acf6d4636a67341c059d66568, user_id=138ec380578e47599b5a78031a412071, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:26.108384 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0354de8-b882-4617-b2ea-9df081572b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:26.110272 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0354de8-b882-4617-b2ea-9df081572b3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:26.110777 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0354de8-b882-4617-b2ea-9df081572b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:26.110963 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200c21d5-ea3d-467d-82cb-d97643ea6d31 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:26.111274 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0354de8-b882-4617-b2ea-9df081572b3c admin admin] PATH_INFO: `/v3/projects/a85913a6ccc24692a25dcdcf0f91de5a` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:26.113626 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0354de8-b882-4617-b2ea-9df081572b3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a85913a6ccc24692a25dcdcf0f91de5a)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:26.117786 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0354de8-b882-4617-b2ea-9df081572b3c admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:26.131431 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1063/2130] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:17:26 2022] PUT /identity/v3/projects/68ad7a3acf6d4636a67341c059d66568/users/138ec380578e47599b5a78031a412071/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:26.145835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc25df-8745-4959-b106-4ac524bb6cc1 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:26.146478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc25df-8745-4959-b106-4ac524bb6cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:26.146898 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc25df-8745-4959-b106-4ac524bb6cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:26.206111 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-a6fc25df-8745-4959-b106-4ac524bb6cc1 None None] MFA Rules not processed for user `138ec380578e47599b5a78031a412071`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:26.207892 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a6fc25df-8745-4959-b106-4ac524bb6cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:26.229894 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-a0354de8-b882-4617-b2ea-9df081572b3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a85913a6ccc24692a25dcdcf0f91de5a', 'request_id': 'req-a0354de8-b882-4617-b2ea-9df081572b3c'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:26.245852 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-a0354de8-b882-4617-b2ea-9df081572b3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a85913a6ccc24692a25dcdcf0f91de5a', 'request_id': 'req-a0354de8-b882-4617-b2ea-9df081572b3c'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:26.263944 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-a0354de8-b882-4617-b2ea-9df081572b3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a85913a6ccc24692a25dcdcf0f91de5a', 'request_id': 'req-a0354de8-b882-4617-b2ea-9df081572b3c'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:26.294081 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-a0354de8-b882-4617-b2ea-9df081572b3c admin admin] The token cache is being invalidate because project a85913a6ccc24692a25dcdcf0f91de5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:17:26.294848 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-a0354de8-b882-4617-b2ea-9df081572b3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:26.296650 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-a0354de8-b882-4617-b2ea-9df081572b3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:26.309612 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1068/2131] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:17:26 2022] DELETE /identity/v3/projects/a85913a6ccc24692a25dcdcf0f91de5a => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:26.312332 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4608eb1-c1ca-4179-94d7-8b1fe9131f5e None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:26.313012 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4608eb1-c1ca-4179-94d7-8b1fe9131f5e None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:26.313761 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4608eb1-c1ca-4179-94d7-8b1fe9131f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:26.326012 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:26.363974 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-d4608eb1-c1ca-4179-94d7-8b1fe9131f5e None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:26.366202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-d4608eb1-c1ca-4179-94d7-8b1fe9131f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:26.411727 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1064/2132] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:26.418707 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc25df-8745-4959-b106-4ac524bb6cc1 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:26.423168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a6fc25df-8745-4959-b106-4ac524bb6cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:26.475082 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:26.551603 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1069/2133] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:17:26.566729 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c5a7a-c9c1-46e9-b82e-60c939c1153a None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:26.570100 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c5a7a-c9c1-46e9-b82e-60c939c1153a None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:26.572115 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c5a7a-c9c1-46e9-b82e-60c939c1153a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:26.594561 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc25df-8745-4959-b106-4ac524bb6cc1 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:26.604601 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-b97c5a7a-c9c1-46e9-b82e-60c939c1153a None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:26.606322 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-b97c5a7a-c9c1-46e9-b82e-60c939c1153a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:26.607687 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59971148-8f2b-4f90-963d-4104a578b36c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:26.609300 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59971148-8f2b-4f90-963d-4104a578b36c service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:26.610044 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59971148-8f2b-4f90-963d-4104a578b36c service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:26.610782 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59971148-8f2b-4f90-963d-4104a578b36c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:26.637206 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1070/2134] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:26.641960 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c5a7a-c9c1-46e9-b82e-60c939c1153a None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:26.645070 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59971148-8f2b-4f90-963d-4104a578b36c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:26.648215 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59971148-8f2b-4f90-963d-4104a578b36c service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:26.671181 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1065/2135] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:17:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:17:26.675616 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c5a7a-c9c1-46e9-b82e-60c939c1153a None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:26.699403 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3eb747-061b-4adf-b99b-198f710d556a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:26.700418 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3eb747-061b-4adf-b99b-198f710d556a admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:26.700866 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3eb747-061b-4adf-b99b-198f710d556a admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:26.701156 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3eb747-061b-4adf-b99b-198f710d556a admin admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:26.702505 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3eb747-061b-4adf-b99b-198f710d556a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:17:26.702505 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3eb747-061b-4adf-b99b-198f710d556a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:26.705163 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3eb747-061b-4adf-b99b-198f710d556a admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:26.726193 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1071/2136] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:17:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:17:26.731889 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59971148-8f2b-4f90-963d-4104a578b36c service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:26.758447 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59971148-8f2b-4f90-963d-4104a578b36c service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:26.771838 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd091a-ca1e-4d5f-b0e6-5f3d063484b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:26.774494 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd091a-ca1e-4d5f-b0e6-5f3d063484b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:26.776094 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd091a-ca1e-4d5f-b0e6-5f3d063484b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:26.776405 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd091a-ca1e-4d5f-b0e6-5f3d063484b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:26.777160 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dd091a-ca1e-4d5f-b0e6-5f3d063484b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:26.779346 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dd091a-ca1e-4d5f-b0e6-5f3d063484b6 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:26.842935 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1066/2137] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:17:26 2022] POST /identity/v3/projects => generated 367 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:17:26.846841 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3eb747-061b-4adf-b99b-198f710d556a admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:26.862215 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3eb747-061b-4adf-b99b-198f710d556a admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:27.032206 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4997a-1b1f-428e-becc-632a960c0683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:27.033101 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4997a-1b1f-428e-becc-632a960c0683 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:27.033471 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4997a-1b1f-428e-becc-632a960c0683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:27.033800 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4997a-1b1f-428e-becc-632a960c0683 admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:27.035862 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c4997a-1b1f-428e-becc-632a960c0683 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:27.042556 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c4997a-1b1f-428e-becc-632a960c0683 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:27.099791 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1072/2138] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:17:26 2022] POST /identity/v3/users => generated 284 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:17:27.103806 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd091a-ca1e-4d5f-b0e6-5f3d063484b6 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:27.119125 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd091a-ca1e-4d5f-b0e6-5f3d063484b6 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:27.133392 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3470dbf-02a9-4661-b6d2-6f52d630225d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:27.137586 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3470dbf-02a9-4661-b6d2-6f52d630225d admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:27.137928 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3470dbf-02a9-4661-b6d2-6f52d630225d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:27.138287 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3470dbf-02a9-4661-b6d2-6f52d630225d admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:27.140731 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3470dbf-02a9-4661-b6d2-6f52d630225d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:27.143311 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3470dbf-02a9-4661-b6d2-6f52d630225d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:27.181183 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1067/2139] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:17:27 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:17:27.184899 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4997a-1b1f-428e-becc-632a960c0683 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:27.203182 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4997a-1b1f-428e-becc-632a960c0683 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:27.212957 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb62787-c52d-4372-a96d-d025728c2488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:27.214121 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb62787-c52d-4372-a96d-d025728c2488 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:27.214636 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb62787-c52d-4372-a96d-d025728c2488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:27.215150 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb62787-c52d-4372-a96d-d025728c2488 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:27.216916 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb62787-c52d-4372-a96d-d025728c2488 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:27.221646 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb62787-c52d-4372-a96d-d025728c2488 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:27.245318 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1073/2140] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:17:27 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:17:27.259498 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3470dbf-02a9-4661-b6d2-6f52d630225d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:27.280187 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3470dbf-02a9-4661-b6d2-6f52d630225d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:27.292125 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a3e128-4ef1-4382-92d1-481d58350231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:27.293213 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a3e128-4ef1-4382-92d1-481d58350231 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:27.293507 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a3e128-4ef1-4382-92d1-481d58350231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:27.293762 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a3e128-4ef1-4382-92d1-481d58350231 admin admin] PATH_INFO: `/v3/projects/c463dbd7f2d44ef8a3703de9af1b6263/users/4365dbae88044ae29ac42a83a519faa1/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:27.338873 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a3e128-4ef1-4382-92d1-481d58350231 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c463dbd7f2d44ef8a3703de9af1b6263, user_id=4365dbae88044ae29ac42a83a519faa1, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:27.342272 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a3e128-4ef1-4382-92d1-481d58350231 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:27.378725 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1068/2141] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:17:27 2022] PUT /identity/v3/projects/c463dbd7f2d44ef8a3703de9af1b6263/users/4365dbae88044ae29ac42a83a519faa1/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:27.391179 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e9ba8-56d2-46d9-8468-4328a853253c None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:27.391743 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e9ba8-56d2-46d9-8468-4328a853253c None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:27.392251 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e9ba8-56d2-46d9-8468-4328a853253c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:27.450335 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-d93e9ba8-56d2-46d9-8468-4328a853253c None None] MFA Rules not processed for user `4365dbae88044ae29ac42a83a519faa1`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:27.452720 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-d93e9ba8-56d2-46d9-8468-4328a853253c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:27.604846 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1074/2142] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:27 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:27.607100 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a3e128-4ef1-4382-92d1-481d58350231 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:27.718676 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a3e128-4ef1-4382-92d1-481d58350231 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:27.734206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af41092-3638-41ee-938a-bf429ee01beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:27.735507 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af41092-3638-41ee-938a-bf429ee01beb admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:27.735929 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af41092-3638-41ee-938a-bf429ee01beb admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:27.736303 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af41092-3638-41ee-938a-bf429ee01beb admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:27.738803 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af41092-3638-41ee-938a-bf429ee01beb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:27.740980 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af41092-3638-41ee-938a-bf429ee01beb admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:27.808869 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1069/2143] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:17:27 2022] POST /identity/v3/projects => generated 369 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:17:27.813866 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93e9ba8-56d2-46d9-8468-4328a853253c None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:27.840628 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93e9ba8-56d2-46d9-8468-4328a853253c None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:27.939374 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:28.006175 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73101c-1a8b-4e9a-91eb-68a09486739a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:28.007326 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73101c-1a8b-4e9a-91eb-68a09486739a admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:28.007863 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73101c-1a8b-4e9a-91eb-68a09486739a admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:28.008359 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd73101c-1a8b-4e9a-91eb-68a09486739a admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:28.009849 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd73101c-1a8b-4e9a-91eb-68a09486739a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:28.013647 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd73101c-1a8b-4e9a-91eb-68a09486739a admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:28.064801 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1075/2144] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:17:27 2022] POST /identity/v3/users => generated 291 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:17:28.069649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af41092-3638-41ee-938a-bf429ee01beb admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:28.090407 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af41092-3638-41ee-938a-bf429ee01beb admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:28.110033 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68d7aa6-5ddc-4388-b26d-a4c0ff76f8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:28.111134 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d7aa6-5ddc-4388-b26d-a4c0ff76f8a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:28.111664 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d7aa6-5ddc-4388-b26d-a4c0ff76f8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:28.112132 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d7aa6-5ddc-4388-b26d-a4c0ff76f8a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:28.113508 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68d7aa6-5ddc-4388-b26d-a4c0ff76f8a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:28.116542 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68d7aa6-5ddc-4388-b26d-a4c0ff76f8a9 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:28.128920 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:28.145578 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1070/2145] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:17:28 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:17:28.156978 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73101c-1a8b-4e9a-91eb-68a09486739a admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:28.159165 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7339944-e4ab-4528-bc85-4a850b670eb6 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:28.159858 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7339944-e4ab-4528-bc85-4a850b670eb6 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:28.160188 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7339944-e4ab-4528-bc85-4a850b670eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:28.186082 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd73101c-1a8b-4e9a-91eb-68a09486739a admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:28.200796 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189bcd4-f30e-4b58-81b1-1b5b4fa91c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:28.202668 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189bcd4-f30e-4b58-81b1-1b5b4fa91c09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:28.206581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189bcd4-f30e-4b58-81b1-1b5b4fa91c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:28.207375 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189bcd4-f30e-4b58-81b1-1b5b4fa91c09 admin admin] PATH_INFO: `/v3/projects/09584555497045e9a69917f1a07c237c/users/13dc2041325f446497d8d16312f8ca76/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:28.224701 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c7339944-e4ab-4528-bc85-4a850b670eb6 None None] MFA Rules not processed for user `2453702c7e11464d8aefea3614549ae9`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:28.227469 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c7339944-e4ab-4528-bc85-4a850b670eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:28.283662 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5189bcd4-f30e-4b58-81b1-1b5b4fa91c09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09584555497045e9a69917f1a07c237c, user_id=13dc2041325f446497d8d16312f8ca76, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:28.294802 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5189bcd4-f30e-4b58-81b1-1b5b4fa91c09 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:28.331442 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1076/2146] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:17:28 2022] PUT /identity/v3/projects/09584555497045e9a69917f1a07c237c/users/13dc2041325f446497d8d16312f8ca76/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:28.336307 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189bcd4-f30e-4b58-81b1-1b5b4fa91c09 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:28.447888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1071/2147] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:28 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:28.450911 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff739a-397c-4dcd-870a-dc75005b32dc None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:28.451448 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff739a-397c-4dcd-870a-dc75005b32dc None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:28.452080 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff739a-397c-4dcd-870a-dc75005b32dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:28.481032 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189bcd4-f30e-4b58-81b1-1b5b4fa91c09 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:28.486502 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db99cac-2bd3-4b27-9141-dec5b290f581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:28.487201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db99cac-2bd3-4b27-9141-dec5b290f581 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:28.487486 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db99cac-2bd3-4b27-9141-dec5b290f581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:28.487755 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-5aff739a-397c-4dcd-870a-dc75005b32dc None None] MFA Rules not processed for user `138ec380578e47599b5a78031a412071`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:28.487939 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db99cac-2bd3-4b27-9141-dec5b290f581 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:28.488661 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db99cac-2bd3-4b27-9141-dec5b290f581 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:28.489868 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5aff739a-397c-4dcd-870a-dc75005b32dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:28.490354 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db99cac-2bd3-4b27-9141-dec5b290f581 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:28.507273 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1077/2148] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:17:28 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:17:28.510162 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db99cac-2bd3-4b27-9141-dec5b290f581 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:28.511926 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-0db99cac-2bd3-4b27-9141-dec5b290f581 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:28.596456 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1072/2149] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:28.597229 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff739a-397c-4dcd-870a-dc75005b32dc None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:28.610859 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff739a-397c-4dcd-870a-dc75005b32dc None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:28.619500 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cadb04-444e-4f22-998f-3705a442f58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:28.620500 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cadb04-444e-4f22-998f-3705a442f58c admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:28.620818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cadb04-444e-4f22-998f-3705a442f58c admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:28.621133 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cadb04-444e-4f22-998f-3705a442f58c admin admin] PATH_INFO: `/v3/projects/09584555497045e9a69917f1a07c237c/users/13dc2041325f446497d8d16312f8ca76/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:28.623477 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cadb04-444e-4f22-998f-3705a442f58c admin admin] RBAC: Authorizing `identity:create_grant(project_id=09584555497045e9a69917f1a07c237c, user_id=13dc2041325f446497d8d16312f8ca76, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:28.626455 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cadb04-444e-4f22-998f-3705a442f58c admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:28.644880 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db99cac-2bd3-4b27-9141-dec5b290f581 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:28.646247 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1073/2150] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:17:28 2022] PUT /identity/v3/projects/09584555497045e9a69917f1a07c237c/users/13dc2041325f446497d8d16312f8ca76/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:28.657803 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8b4bd-b712-410b-85be-4547eae4cb23 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:28.658135 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8b4bd-b712-410b-85be-4547eae4cb23 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:28.658408 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8b4bd-b712-410b-85be-4547eae4cb23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:28.706357 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c0c8b4bd-b712-410b-85be-4547eae4cb23 None None] MFA Rules not processed for user `13dc2041325f446497d8d16312f8ca76`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:28.709088 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c0c8b4bd-b712-410b-85be-4547eae4cb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:28.727510 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:28.796519 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34ad02-6752-49dc-b066-6a6b70ef4b75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:28.797726 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34ad02-6752-49dc-b066-6a6b70ef4b75 service nova] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:28.798178 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34ad02-6752-49dc-b066-6a6b70ef4b75 service nova] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:28.798609 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34ad02-6752-49dc-b066-6a6b70ef4b75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:28.802055 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-bd34ad02-6752-49dc-b066-6a6b70ef4b75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:28.878486 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:28.894661 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1074/2151] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:28 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:28.912303 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c8b4bd-b712-410b-85be-4547eae4cb23 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:28.921555 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c0c8b4bd-b712-410b-85be-4547eae4cb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:28.945039 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd34ad02-6752-49dc-b066-6a6b70ef4b75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:28.963889 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd34ad02-6752-49dc-b066-6a6b70ef4b75 service nova] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:28.999041 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1078/2152] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:17:28 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:17:29.087172 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c8b4bd-b712-410b-85be-4547eae4cb23 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:29.095261 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d5dcd-a747-41b9-8323-945e96a559a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:29.096714 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d5dcd-a747-41b9-8323-945e96a559a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:29.097225 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d5dcd-a747-41b9-8323-945e96a559a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:29.097745 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d5dcd-a747-41b9-8323-945e96a559a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:29.124240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6d5dcd-a747-41b9-8323-945e96a559a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:29.127372 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6d5dcd-a747-41b9-8323-945e96a559a4 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:29.161054 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1075/2153] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:17:28 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:17:29.237039 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34ad02-6752-49dc-b066-6a6b70ef4b75 service nova] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:29.251589 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34ad02-6752-49dc-b066-6a6b70ef4b75 service nova] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:29.275319 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5570ad-e1f6-4a79-9fda-d85abbd1fa8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:29.276555 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5570ad-e1f6-4a79-9fda-d85abbd1fa8d service cinder] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:29.277074 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5570ad-e1f6-4a79-9fda-d85abbd1fa8d service cinder] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:29.277614 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5570ad-e1f6-4a79-9fda-d85abbd1fa8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:29.427966 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5570ad-e1f6-4a79-9fda-d85abbd1fa8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:29.432794 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5570ad-e1f6-4a79-9fda-d85abbd1fa8d service cinder] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:29.459184 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1079/2154] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:17:29 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:17:34.183151 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d5dcd-a747-41b9-8323-945e96a559a4 service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:34.348293 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d5dcd-a747-41b9-8323-945e96a559a4 service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:34.372451 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d53f64-650a-4fe3-9ec3-381205d8f7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:34.373683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d53f64-650a-4fe3-9ec3-381205d8f7a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:34.374063 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d53f64-650a-4fe3-9ec3-381205d8f7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:34.374408 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d53f64-650a-4fe3-9ec3-381205d8f7a9 admin admin] PATH_INFO: `/v3/users/4365dbae88044ae29ac42a83a519faa1` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:34.381625 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d53f64-650a-4fe3-9ec3-381205d8f7a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4365dbae88044ae29ac42a83a519faa1)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:34.385330 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d53f64-650a-4fe3-9ec3-381205d8f7a9 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:34.487586 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-76d53f64-650a-4fe3-9ec3-381205d8f7a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4365dbae88044ae29ac42a83a519faa1'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:34.493099 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-76d53f64-650a-4fe3-9ec3-381205d8f7a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4365dbae88044ae29ac42a83a519faa1'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:34.547009 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-76d53f64-650a-4fe3-9ec3-381205d8f7a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4365dbae88044ae29ac42a83a519faa1'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:34.588157 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-76d53f64-650a-4fe3-9ec3-381205d8f7a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4365dbae88044ae29ac42a83a519faa1'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:34.614615 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-76d53f64-650a-4fe3-9ec3-381205d8f7a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4365dbae88044ae29ac42a83a519faa1'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:34.639597 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1076/2155] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:17:34 2022] DELETE /identity/v3/users/4365dbae88044ae29ac42a83a519faa1 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:34.644149 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90820d74-a17d-47c9-8e4d-a424e2a7d3e4 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:34.650013 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90820d74-a17d-47c9-8e4d-a424e2a7d3e4 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:34.651961 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90820d74-a17d-47c9-8e4d-a424e2a7d3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:34.748689 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-90820d74-a17d-47c9-8e4d-a424e2a7d3e4 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:34.750988 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-90820d74-a17d-47c9-8e4d-a424e2a7d3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:35.027500 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1080/2156] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:35.045019 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d53f64-650a-4fe3-9ec3-381205d8f7a9 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:35.047681 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-76d53f64-650a-4fe3-9ec3-381205d8f7a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:35.132403 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:35.214777 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d53f64-650a-4fe3-9ec3-381205d8f7a9 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:35.219642 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd84ba31-0e8b-416e-acae-5b051a184c3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:35.220622 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd84ba31-0e8b-416e-acae-5b051a184c3a service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:35.220988 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd84ba31-0e8b-416e-acae-5b051a184c3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:35.221313 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd84ba31-0e8b-416e-acae-5b051a184c3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:35.233650 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd84ba31-0e8b-416e-acae-5b051a184c3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:35.235242 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd84ba31-0e8b-416e-acae-5b051a184c3a service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:35.247621 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1077/2157] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:17:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:17:35.748894 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90820d74-a17d-47c9-8e4d-a424e2a7d3e4 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:35.751404 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-90820d74-a17d-47c9-8e4d-a424e2a7d3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:35.784128 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90820d74-a17d-47c9-8e4d-a424e2a7d3e4 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:35.798185 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09efc91-623f-4c56-8893-e9925e2c69c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:35.801082 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09efc91-623f-4c56-8893-e9925e2c69c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:35.802448 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09efc91-623f-4c56-8893-e9925e2c69c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:35.803150 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09efc91-623f-4c56-8893-e9925e2c69c1 admin admin] PATH_INFO: `/v3/projects/c463dbd7f2d44ef8a3703de9af1b6263` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:35.805461 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09efc91-623f-4c56-8893-e9925e2c69c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c463dbd7f2d44ef8a3703de9af1b6263)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:35.808646 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09efc91-623f-4c56-8893-e9925e2c69c1 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:35.815227 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:35.900369 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-b09efc91-623f-4c56-8893-e9925e2c69c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c463dbd7f2d44ef8a3703de9af1b6263', 'request_id': 'req-b09efc91-623f-4c56-8893-e9925e2c69c1'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:35.913354 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-b09efc91-623f-4c56-8893-e9925e2c69c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c463dbd7f2d44ef8a3703de9af1b6263', 'request_id': 'req-b09efc91-623f-4c56-8893-e9925e2c69c1'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:35.929401 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-b09efc91-623f-4c56-8893-e9925e2c69c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c463dbd7f2d44ef8a3703de9af1b6263', 'request_id': 'req-b09efc91-623f-4c56-8893-e9925e2c69c1'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:35.951781 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-b09efc91-623f-4c56-8893-e9925e2c69c1 admin admin] The token cache is being invalidate because project c463dbd7f2d44ef8a3703de9af1b6263 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:17:35.952585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-b09efc91-623f-4c56-8893-e9925e2c69c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:35.953784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-b09efc91-623f-4c56-8893-e9925e2c69c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:35.958869 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1081/2158] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:17:35 2022] DELETE /identity/v3/projects/c463dbd7f2d44ef8a3703de9af1b6263 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:35.964834 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd84ba31-0e8b-416e-acae-5b051a184c3a service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:35.979099 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd84ba31-0e8b-416e-acae-5b051a184c3a service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:36.056195 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba8580-f05c-41f4-8ebe-d6cd702c2ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:36.057082 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba8580-f05c-41f4-8ebe-d6cd702c2ca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:36.057369 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba8580-f05c-41f4-8ebe-d6cd702c2ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:36.057637 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba8580-f05c-41f4-8ebe-d6cd702c2ca5 admin admin] PATH_INFO: `/v3/users/13dc2041325f446497d8d16312f8ca76` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:36.058896 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba8580-f05c-41f4-8ebe-d6cd702c2ca5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13dc2041325f446497d8d16312f8ca76)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:36.060887 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba8580-f05c-41f4-8ebe-d6cd702c2ca5 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:36.156923 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d4ba8580-f05c-41f4-8ebe-d6cd702c2ca5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13dc2041325f446497d8d16312f8ca76'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:36.158655 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d4ba8580-f05c-41f4-8ebe-d6cd702c2ca5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13dc2041325f446497d8d16312f8ca76'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:36.189470 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d4ba8580-f05c-41f4-8ebe-d6cd702c2ca5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13dc2041325f446497d8d16312f8ca76'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:36.199187 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d4ba8580-f05c-41f4-8ebe-d6cd702c2ca5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13dc2041325f446497d8d16312f8ca76'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:36.209822 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d4ba8580-f05c-41f4-8ebe-d6cd702c2ca5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13dc2041325f446497d8d16312f8ca76'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:36.219342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1078/2159] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:17:35 2022] DELETE /identity/v3/users/13dc2041325f446497d8d16312f8ca76 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:36.768329 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09efc91-623f-4c56-8893-e9925e2c69c1 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:36.772630 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-b09efc91-623f-4c56-8893-e9925e2c69c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:36.828891 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09efc91-623f-4c56-8893-e9925e2c69c1 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:36.938776 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfa11d4-3f8a-42a4-bfbe-fa81a9f07500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:36.940402 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfa11d4-3f8a-42a4-bfbe-fa81a9f07500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:36.940825 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfa11d4-3f8a-42a4-bfbe-fa81a9f07500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:36.941328 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfa11d4-3f8a-42a4-bfbe-fa81a9f07500 admin admin] PATH_INFO: `/v3/projects/09584555497045e9a69917f1a07c237c` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:36.943737 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfa11d4-3f8a-42a4-bfbe-fa81a9f07500 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09584555497045e9a69917f1a07c237c)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:36.947447 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfa11d4-3f8a-42a4-bfbe-fa81a9f07500 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:37.034695 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-3dfa11d4-3f8a-42a4-bfbe-fa81a9f07500 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09584555497045e9a69917f1a07c237c', 'request_id': 'req-3dfa11d4-3f8a-42a4-bfbe-fa81a9f07500'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:37.043000 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-3dfa11d4-3f8a-42a4-bfbe-fa81a9f07500 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09584555497045e9a69917f1a07c237c', 'request_id': 'req-3dfa11d4-3f8a-42a4-bfbe-fa81a9f07500'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:37.057133 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-3dfa11d4-3f8a-42a4-bfbe-fa81a9f07500 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09584555497045e9a69917f1a07c237c', 'request_id': 'req-3dfa11d4-3f8a-42a4-bfbe-fa81a9f07500'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:37.076877 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-3dfa11d4-3f8a-42a4-bfbe-fa81a9f07500 admin admin] The token cache is being invalidate because project 09584555497045e9a69917f1a07c237c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:17:37.077432 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-3dfa11d4-3f8a-42a4-bfbe-fa81a9f07500 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:37.078212 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-3dfa11d4-3f8a-42a4-bfbe-fa81a9f07500 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:37.090579 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1082/2160] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:17:36 2022] DELETE /identity/v3/projects/09584555497045e9a69917f1a07c237c => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:37.093415 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aafbcd-102d-446c-8f6a-1d13d90018f0 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:37.094261 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aafbcd-102d-446c-8f6a-1d13d90018f0 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:37.094983 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aafbcd-102d-446c-8f6a-1d13d90018f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:37.137832 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-75aafbcd-102d-446c-8f6a-1d13d90018f0 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:37.139402 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-75aafbcd-102d-446c-8f6a-1d13d90018f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:37.205955 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:37.246152 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1079/2161] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:37.259800 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70eb84-68a0-4c0b-9fb1-86ff4dcd4cc6 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:37.260086 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70eb84-68a0-4c0b-9fb1-86ff4dcd4cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:37.260364 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70eb84-68a0-4c0b-9fb1-86ff4dcd4cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:37.281050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-6a70eb84-68a0-4c0b-9fb1-86ff4dcd4cc6 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:37.282506 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-6a70eb84-68a0-4c0b-9fb1-86ff4dcd4cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:37.291550 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1083/2162] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:17:37.297948 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aafbcd-102d-446c-8f6a-1d13d90018f0 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:37.314063 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aafbcd-102d-446c-8f6a-1d13d90018f0 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:37.322956 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c3ace9-31dd-490f-967c-dba0dfe4bc8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:37.324414 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c3ace9-31dd-490f-967c-dba0dfe4bc8a admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:37.324863 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c3ace9-31dd-490f-967c-dba0dfe4bc8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:37.325361 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c3ace9-31dd-490f-967c-dba0dfe4bc8a admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:37.326663 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c3ace9-31dd-490f-967c-dba0dfe4bc8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:17:37.327204 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c3ace9-31dd-490f-967c-dba0dfe4bc8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:37.331080 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c3ace9-31dd-490f-967c-dba0dfe4bc8a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:37.352598 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1080/2163] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:17:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:17:37.356918 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a70eb84-68a0-4c0b-9fb1-86ff4dcd4cc6 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:37.370122 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a70eb84-68a0-4c0b-9fb1-86ff4dcd4cc6 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:37.378111 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf711d-438e-4713-a185-3346e049620c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:37.379255 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf711d-438e-4713-a185-3346e049620c admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:37.379819 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf711d-438e-4713-a185-3346e049620c admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:37.380308 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf711d-438e-4713-a185-3346e049620c admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:37.381540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edf711d-438e-4713-a185-3346e049620c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:37.384925 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edf711d-438e-4713-a185-3346e049620c admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:37.402627 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:37.439992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1084/2164] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:17:37 2022] POST /identity/v3/projects => generated 381 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:17:37.444553 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c3ace9-31dd-490f-967c-dba0dfe4bc8a admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:37.455578 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c3ace9-31dd-490f-967c-dba0dfe4bc8a admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:37.518805 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7c0ea-8547-4e1b-84ae-6d755a3a023a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:37.519490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7c0ea-8547-4e1b-84ae-6d755a3a023a admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:37.519808 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7c0ea-8547-4e1b-84ae-6d755a3a023a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:37.520086 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7c0ea-8547-4e1b-84ae-6d755a3a023a admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:37.520798 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b7c0ea-8547-4e1b-84ae-6d755a3a023a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:37.522766 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b7c0ea-8547-4e1b-84ae-6d755a3a023a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:37.549396 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1081/2165] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:17:37 2022] POST /identity/v3/users => generated 291 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:17:37.554352 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf711d-438e-4713-a185-3346e049620c admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:37.568773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf711d-438e-4713-a185-3346e049620c admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:37.574427 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f68ba6-f628-4f03-b157-2b060a425eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:37.575109 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f68ba6-f628-4f03-b157-2b060a425eb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:37.575390 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f68ba6-f628-4f03-b157-2b060a425eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:37.575704 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f68ba6-f628-4f03-b157-2b060a425eb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:37.576487 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f68ba6-f628-4f03-b157-2b060a425eb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:37.578327 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f68ba6-f628-4f03-b157-2b060a425eb5 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:37.590552 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1085/2166] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:17:37.596642 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7c0ea-8547-4e1b-84ae-6d755a3a023a admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:37.607624 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7c0ea-8547-4e1b-84ae-6d755a3a023a admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:37.617373 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7515699-4986-4b9b-af15-a7dfae2505d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:37.618472 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7515699-4986-4b9b-af15-a7dfae2505d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:37.618986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7515699-4986-4b9b-af15-a7dfae2505d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:37.619338 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7515699-4986-4b9b-af15-a7dfae2505d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:37.620206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7515699-4986-4b9b-af15-a7dfae2505d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:37.622012 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7515699-4986-4b9b-af15-a7dfae2505d5 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:37.636229 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1082/2167] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:17:37.641630 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f68ba6-f628-4f03-b157-2b060a425eb5 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:37.656436 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f68ba6-f628-4f03-b157-2b060a425eb5 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:37.667325 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d19164a-e701-4473-bd94-5f70d1692c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:37.668418 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19164a-e701-4473-bd94-5f70d1692c7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:37.668805 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19164a-e701-4473-bd94-5f70d1692c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:37.669659 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19164a-e701-4473-bd94-5f70d1692c7e admin admin] PATH_INFO: `/v3/projects/db29ded4c53b41339cbc2415016de344/users/76d03320efa142e4bea775924236cc91/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:37.689659 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d19164a-e701-4473-bd94-5f70d1692c7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=db29ded4c53b41339cbc2415016de344, user_id=76d03320efa142e4bea775924236cc91, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:37.692997 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d19164a-e701-4473-bd94-5f70d1692c7e admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:37.708348 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1086/2168] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:17:37 2022] PUT /identity/v3/projects/db29ded4c53b41339cbc2415016de344/users/76d03320efa142e4bea775924236cc91/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:17:37.719569 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba71962-102e-45da-8bcd-fb9e79bc45c6 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:37.719871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba71962-102e-45da-8bcd-fb9e79bc45c6 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:37.720174 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba71962-102e-45da-8bcd-fb9e79bc45c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:37.751099 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-4ba71962-102e-45da-8bcd-fb9e79bc45c6 None None] MFA Rules not processed for user `76d03320efa142e4bea775924236cc91`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:37.752662 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4ba71962-102e-45da-8bcd-fb9e79bc45c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:37.828723 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1083/2169] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:37.834116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d19164a-e701-4473-bd94-5f70d1692c7e admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:37.922790 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d19164a-e701-4473-bd94-5f70d1692c7e admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:37.929308 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc134c2b-28eb-47fc-9587-1a0919dba6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:37.930147 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc134c2b-28eb-47fc-9587-1a0919dba6f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:37.930525 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc134c2b-28eb-47fc-9587-1a0919dba6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:37.930914 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc134c2b-28eb-47fc-9587-1a0919dba6f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:37.931756 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc134c2b-28eb-47fc-9587-1a0919dba6f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:37.933867 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc134c2b-28eb-47fc-9587-1a0919dba6f1 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:37.962189 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1087/2170] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:17:37 2022] POST /identity/v3/projects => generated 379 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:17:37.968307 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba71962-102e-45da-8bcd-fb9e79bc45c6 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:37.984942 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba71962-102e-45da-8bcd-fb9e79bc45c6 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:38.051459 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:38.072894 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf225a-6566-41e0-8879-8d79a084e2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:38.073613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf225a-6566-41e0-8879-8d79a084e2f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:38.073895 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf225a-6566-41e0-8879-8d79a084e2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:38.074165 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf225a-6566-41e0-8879-8d79a084e2f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:38.074872 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bf225a-6566-41e0-8879-8d79a084e2f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:38.076938 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bf225a-6566-41e0-8879-8d79a084e2f7 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:38.098798 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1084/2171] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:17:37 2022] POST /identity/v3/users => generated 296 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:17:38.103884 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc134c2b-28eb-47fc-9587-1a0919dba6f1 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:38.116274 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc134c2b-28eb-47fc-9587-1a0919dba6f1 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:38.121221 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dfddb3-ba51-4880-8ca5-3bbf6f73895c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:38.121926 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dfddb3-ba51-4880-8ca5-3bbf6f73895c admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:38.122229 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dfddb3-ba51-4880-8ca5-3bbf6f73895c admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:38.122505 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dfddb3-ba51-4880-8ca5-3bbf6f73895c admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:38.123297 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dfddb3-ba51-4880-8ca5-3bbf6f73895c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:38.124960 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dfddb3-ba51-4880-8ca5-3bbf6f73895c admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:38.140126 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1088/2172] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:17:38.145722 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf225a-6566-41e0-8879-8d79a084e2f7 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:38.164636 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf225a-6566-41e0-8879-8d79a084e2f7 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:38.173892 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195698ea-433f-4620-adc3-f302dc0ae773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:38.175298 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195698ea-433f-4620-adc3-f302dc0ae773 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:38.175826 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195698ea-433f-4620-adc3-f302dc0ae773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:38.176204 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195698ea-433f-4620-adc3-f302dc0ae773 admin admin] PATH_INFO: `/v3/projects/0b01e452546e407fb37a564e36433c26/users/ba8b741905ea47bdb29268a4d2e7b1b0/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:38.206460 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195698ea-433f-4620-adc3-f302dc0ae773 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b01e452546e407fb37a564e36433c26, user_id=ba8b741905ea47bdb29268a4d2e7b1b0, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:38.212445 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195698ea-433f-4620-adc3-f302dc0ae773 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:38.234395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1085/2173] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:17:38 2022] PUT /identity/v3/projects/0b01e452546e407fb37a564e36433c26/users/ba8b741905ea47bdb29268a4d2e7b1b0/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:38.238693 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dfddb3-ba51-4880-8ca5-3bbf6f73895c admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:38.250865 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dfddb3-ba51-4880-8ca5-3bbf6f73895c admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:38.324142 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45739b00-dc0d-4377-9470-9075ed5caaa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:38.325319 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45739b00-dc0d-4377-9470-9075ed5caaa0 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:38.325814 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45739b00-dc0d-4377-9470-9075ed5caaa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:38.326310 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45739b00-dc0d-4377-9470-9075ed5caaa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:38.327725 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45739b00-dc0d-4377-9470-9075ed5caaa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:38.330819 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45739b00-dc0d-4377-9470-9075ed5caaa0 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:38.351143 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1089/2174] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:17:38.355767 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195698ea-433f-4620-adc3-f302dc0ae773 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:38.367153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195698ea-433f-4620-adc3-f302dc0ae773 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:38.373361 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e9865-16fa-4fdd-97ff-6527b5d76496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:38.374280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e9865-16fa-4fdd-97ff-6527b5d76496 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:38.374556 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e9865-16fa-4fdd-97ff-6527b5d76496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:38.374827 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e9865-16fa-4fdd-97ff-6527b5d76496 admin admin] PATH_INFO: `/v3/projects/0b01e452546e407fb37a564e36433c26/users/ba8b741905ea47bdb29268a4d2e7b1b0/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:38.376923 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e9865-16fa-4fdd-97ff-6527b5d76496 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b01e452546e407fb37a564e36433c26, user_id=ba8b741905ea47bdb29268a4d2e7b1b0, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:38.379687 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e9865-16fa-4fdd-97ff-6527b5d76496 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:38.399512 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1086/2175] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:17:38 2022] PUT /identity/v3/projects/0b01e452546e407fb37a564e36433c26/users/ba8b741905ea47bdb29268a4d2e7b1b0/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:38.412297 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53130e29-4c10-4a4e-8c3f-4a56a876d741 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:38.412590 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53130e29-4c10-4a4e-8c3f-4a56a876d741 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:38.412876 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53130e29-4c10-4a4e-8c3f-4a56a876d741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:38.442935 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-53130e29-4c10-4a4e-8c3f-4a56a876d741 None None] MFA Rules not processed for user `ba8b741905ea47bdb29268a4d2e7b1b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:38.444387 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-53130e29-4c10-4a4e-8c3f-4a56a876d741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:38.463648 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:38.533441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1090/2176] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:38 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:38.547568 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e9865-16fa-4fdd-97ff-6527b5d76496 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:38.549764 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-041e9865-16fa-4fdd-97ff-6527b5d76496 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:38.619157 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e9865-16fa-4fdd-97ff-6527b5d76496 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:38.623036 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60359bae-aa2f-4563-91fa-52a45827a045 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:38.623732 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60359bae-aa2f-4563-91fa-52a45827a045 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:38.624020 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60359bae-aa2f-4563-91fa-52a45827a045 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:38.624311 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60359bae-aa2f-4563-91fa-52a45827a045 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:38.635660 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60359bae-aa2f-4563-91fa-52a45827a045 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:38.637175 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60359bae-aa2f-4563-91fa-52a45827a045 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:38.648953 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1087/2177] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:17:38 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:17:39.128360 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53130e29-4c10-4a4e-8c3f-4a56a876d741 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:39.143331 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53130e29-4c10-4a4e-8c3f-4a56a876d741 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:39.149485 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecfb27f-3102-4eb5-902f-0f6518361647 tempest-VolumeQuotaClassesTest-855136721 tempest-VolumeQuotaClassesTest-855136721-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8b741905ea47bdb29268a4d2e7b1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b01e452546e407fb37a564e36433c26', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:39.150179 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecfb27f-3102-4eb5-902f-0f6518361647 tempest-VolumeQuotaClassesTest-855136721 tempest-VolumeQuotaClassesTest-855136721-project-admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:39.150467 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecfb27f-3102-4eb5-902f-0f6518361647 tempest-VolumeQuotaClassesTest-855136721 tempest-VolumeQuotaClassesTest-855136721-project-admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:39.150749 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecfb27f-3102-4eb5-902f-0f6518361647 tempest-VolumeQuotaClassesTest-855136721 tempest-VolumeQuotaClassesTest-855136721-project-admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:39.151516 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecfb27f-3102-4eb5-902f-0f6518361647 tempest-VolumeQuotaClassesTest-855136721 tempest-VolumeQuotaClassesTest-855136721-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:17:39.151862 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecfb27f-3102-4eb5-902f-0f6518361647 tempest-VolumeQuotaClassesTest-855136721 tempest-VolumeQuotaClassesTest-855136721-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:39.153976 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecfb27f-3102-4eb5-902f-0f6518361647 tempest-VolumeQuotaClassesTest-855136721 tempest-VolumeQuotaClassesTest-855136721-project-admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:39.173660 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1091/2178] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:17:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:17:39.180411 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60359bae-aa2f-4563-91fa-52a45827a045 service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:39.191604 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60359bae-aa2f-4563-91fa-52a45827a045 service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:39.198062 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5781e5-48d9-4457-b33d-b6452a163d84 tempest-VolumeQuotaClassesTest-855136721 tempest-VolumeQuotaClassesTest-855136721-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8b741905ea47bdb29268a4d2e7b1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b01e452546e407fb37a564e36433c26', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:39.198835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5781e5-48d9-4457-b33d-b6452a163d84 tempest-VolumeQuotaClassesTest-855136721 tempest-VolumeQuotaClassesTest-855136721-project-admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:39.199155 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5781e5-48d9-4457-b33d-b6452a163d84 tempest-VolumeQuotaClassesTest-855136721 tempest-VolumeQuotaClassesTest-855136721-project-admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:39.199472 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5781e5-48d9-4457-b33d-b6452a163d84 tempest-VolumeQuotaClassesTest-855136721 tempest-VolumeQuotaClassesTest-855136721-project-admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:39.200267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5781e5-48d9-4457-b33d-b6452a163d84 tempest-VolumeQuotaClassesTest-855136721 tempest-VolumeQuotaClassesTest-855136721-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:39.202529 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5781e5-48d9-4457-b33d-b6452a163d84 tempest-VolumeQuotaClassesTest-855136721 tempest-VolumeQuotaClassesTest-855136721-project-admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:39.233929 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1088/2179] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:17:39 2022] POST /identity/v3/projects => generated 354 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:17:39.241179 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecfb27f-3102-4eb5-902f-0f6518361647 tempest-VolumeQuotaClassesTest-855136721 tempest-VolumeQuotaClassesTest-855136721-project-admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:39.261103 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecfb27f-3102-4eb5-902f-0f6518361647 tempest-VolumeQuotaClassesTest-855136721 tempest-VolumeQuotaClassesTest-855136721-project-admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:39.364979 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57776a4b-42ea-49ab-a1f2-fb21e1414fa3 tempest-VolumeQuotaClassesTest-855136721 tempest-VolumeQuotaClassesTest-855136721-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8b741905ea47bdb29268a4d2e7b1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b01e452546e407fb37a564e36433c26', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:39.366101 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57776a4b-42ea-49ab-a1f2-fb21e1414fa3 tempest-VolumeQuotaClassesTest-855136721 tempest-VolumeQuotaClassesTest-855136721-project-admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:39.366585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57776a4b-42ea-49ab-a1f2-fb21e1414fa3 tempest-VolumeQuotaClassesTest-855136721 tempest-VolumeQuotaClassesTest-855136721-project-admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:39.367198 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57776a4b-42ea-49ab-a1f2-fb21e1414fa3 tempest-VolumeQuotaClassesTest-855136721 tempest-VolumeQuotaClassesTest-855136721-project-admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:39.368595 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57776a4b-42ea-49ab-a1f2-fb21e1414fa3 tempest-VolumeQuotaClassesTest-855136721 tempest-VolumeQuotaClassesTest-855136721-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:17:39.369162 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57776a4b-42ea-49ab-a1f2-fb21e1414fa3 tempest-VolumeQuotaClassesTest-855136721 tempest-VolumeQuotaClassesTest-855136721-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:39.374047 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57776a4b-42ea-49ab-a1f2-fb21e1414fa3 tempest-VolumeQuotaClassesTest-855136721 tempest-VolumeQuotaClassesTest-855136721-project-admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:39.404885 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1092/2180] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:17:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:17:39.447820 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5781e5-48d9-4457-b33d-b6452a163d84 tempest-VolumeQuotaClassesTest-855136721 tempest-VolumeQuotaClassesTest-855136721-project-admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:39.465910 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5781e5-48d9-4457-b33d-b6452a163d84 tempest-VolumeQuotaClassesTest-855136721 tempest-VolumeQuotaClassesTest-855136721-project-admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:39.471090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78487fe-6672-4d1e-9850-ac67ba109430 tempest-VolumeQuotaClassesTest-855136721 tempest-VolumeQuotaClassesTest-855136721-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8b741905ea47bdb29268a4d2e7b1b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b01e452546e407fb37a564e36433c26', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:39.472020 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78487fe-6672-4d1e-9850-ac67ba109430 tempest-VolumeQuotaClassesTest-855136721 tempest-VolumeQuotaClassesTest-855136721-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:39.472296 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78487fe-6672-4d1e-9850-ac67ba109430 tempest-VolumeQuotaClassesTest-855136721 tempest-VolumeQuotaClassesTest-855136721-project-admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:39.472573 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78487fe-6672-4d1e-9850-ac67ba109430 tempest-VolumeQuotaClassesTest-855136721 tempest-VolumeQuotaClassesTest-855136721-project-admin] PATH_INFO: `/v3/projects/6401e4b2a0f3476aa6bfc95e6937755d` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:39.473844 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78487fe-6672-4d1e-9850-ac67ba109430 tempest-VolumeQuotaClassesTest-855136721 tempest-VolumeQuotaClassesTest-855136721-project-admin] RBAC: Authorizing `identity:delete_project(project_id=6401e4b2a0f3476aa6bfc95e6937755d)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:39.475815 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78487fe-6672-4d1e-9850-ac67ba109430 tempest-VolumeQuotaClassesTest-855136721 tempest-VolumeQuotaClassesTest-855136721-project-admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:39.535294 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c78487fe-6672-4d1e-9850-ac67ba109430 tempest-VolumeQuotaClassesTest-855136721 tempest-VolumeQuotaClassesTest-855136721-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6401e4b2a0f3476aa6bfc95e6937755d', 'request_id': 'req-c78487fe-6672-4d1e-9850-ac67ba109430'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:39.543362 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c78487fe-6672-4d1e-9850-ac67ba109430 tempest-VolumeQuotaClassesTest-855136721 tempest-VolumeQuotaClassesTest-855136721-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6401e4b2a0f3476aa6bfc95e6937755d', 'request_id': 'req-c78487fe-6672-4d1e-9850-ac67ba109430'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:39.552985 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c78487fe-6672-4d1e-9850-ac67ba109430 tempest-VolumeQuotaClassesTest-855136721 tempest-VolumeQuotaClassesTest-855136721-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6401e4b2a0f3476aa6bfc95e6937755d', 'request_id': 'req-c78487fe-6672-4d1e-9850-ac67ba109430'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:39.559254 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:39.572339 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c78487fe-6672-4d1e-9850-ac67ba109430 tempest-VolumeQuotaClassesTest-855136721 tempest-VolumeQuotaClassesTest-855136721-project-admin] The token cache is being invalidate because project 6401e4b2a0f3476aa6bfc95e6937755d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:17:39.573351 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c78487fe-6672-4d1e-9850-ac67ba109430 tempest-VolumeQuotaClassesTest-855136721 tempest-VolumeQuotaClassesTest-855136721-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:39.574671 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c78487fe-6672-4d1e-9850-ac67ba109430 tempest-VolumeQuotaClassesTest-855136721 tempest-VolumeQuotaClassesTest-855136721-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:39.578014 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1089/2181] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:17:39 2022] DELETE /identity/v3/projects/6401e4b2a0f3476aa6bfc95e6937755d => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:17:39.889694 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57776a4b-42ea-49ab-a1f2-fb21e1414fa3 tempest-VolumeQuotaClassesTest-855136721 tempest-VolumeQuotaClassesTest-855136721-project-admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:39.891518 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-57776a4b-42ea-49ab-a1f2-fb21e1414fa3 tempest-VolumeQuotaClassesTest-855136721 tempest-VolumeQuotaClassesTest-855136721-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:39.993989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57776a4b-42ea-49ab-a1f2-fb21e1414fa3 tempest-VolumeQuotaClassesTest-855136721 tempest-VolumeQuotaClassesTest-855136721-project-admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:40.061305 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:40.095477 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb9071-1b02-474e-9bb1-252ef2630d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:40.096530 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb9071-1b02-474e-9bb1-252ef2630d0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:40.096965 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb9071-1b02-474e-9bb1-252ef2630d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:40.097335 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb9071-1b02-474e-9bb1-252ef2630d0f admin admin] PATH_INFO: `/v3/users/76d03320efa142e4bea775924236cc91` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:40.099092 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb9071-1b02-474e-9bb1-252ef2630d0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=76d03320efa142e4bea775924236cc91)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:40.101469 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb9071-1b02-474e-9bb1-252ef2630d0f admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:40.168583 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-d9fb9071-1b02-474e-9bb1-252ef2630d0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76d03320efa142e4bea775924236cc91'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:40.170069 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-d9fb9071-1b02-474e-9bb1-252ef2630d0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76d03320efa142e4bea775924236cc91'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:40.195940 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-d9fb9071-1b02-474e-9bb1-252ef2630d0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76d03320efa142e4bea775924236cc91'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:40.216355 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-d9fb9071-1b02-474e-9bb1-252ef2630d0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76d03320efa142e4bea775924236cc91'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:40.217443 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-d9fb9071-1b02-474e-9bb1-252ef2630d0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76d03320efa142e4bea775924236cc91'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:40.233409 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1093/2182] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:17:39 2022] DELETE /identity/v3/users/76d03320efa142e4bea775924236cc91 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:17:40.238285 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742bdb5-b22b-44a7-9b5d-30feb406bc57 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:40.238913 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742bdb5-b22b-44a7-9b5d-30feb406bc57 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:40.239330 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742bdb5-b22b-44a7-9b5d-30feb406bc57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:40.264389 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-d742bdb5-b22b-44a7-9b5d-30feb406bc57 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:40.266205 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d742bdb5-b22b-44a7-9b5d-30feb406bc57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:40.364799 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1090/2183] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:40.382330 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb9071-1b02-474e-9bb1-252ef2630d0f admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:40.385198 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-d9fb9071-1b02-474e-9bb1-252ef2630d0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:40.471391 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb9071-1b02-474e-9bb1-252ef2630d0f admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:40.476412 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e552d0-974b-4a49-9543-99b789af2b39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:40.478009 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e552d0-974b-4a49-9543-99b789af2b39 service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:40.478296 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e552d0-974b-4a49-9543-99b789af2b39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:40.478626 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e552d0-974b-4a49-9543-99b789af2b39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:40.500504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e552d0-974b-4a49-9543-99b789af2b39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:40.502620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e552d0-974b-4a49-9543-99b789af2b39 service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:40.514210 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1094/2184] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:17:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:17:41.234201 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d742bdb5-b22b-44a7-9b5d-30feb406bc57 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:41.237976 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d742bdb5-b22b-44a7-9b5d-30feb406bc57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:41.264333 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d742bdb5-b22b-44a7-9b5d-30feb406bc57 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:41.278614 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08986d7b-a27e-40ea-a1c2-f0d9d98ab3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:41.281735 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08986d7b-a27e-40ea-a1c2-f0d9d98ab3b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:41.282453 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08986d7b-a27e-40ea-a1c2-f0d9d98ab3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:41.282893 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08986d7b-a27e-40ea-a1c2-f0d9d98ab3b4 admin admin] PATH_INFO: `/v3/projects/db29ded4c53b41339cbc2415016de344` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:41.286135 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08986d7b-a27e-40ea-a1c2-f0d9d98ab3b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db29ded4c53b41339cbc2415016de344)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:41.289683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08986d7b-a27e-40ea-a1c2-f0d9d98ab3b4 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:41.389659 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-08986d7b-a27e-40ea-a1c2-f0d9d98ab3b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db29ded4c53b41339cbc2415016de344', 'request_id': 'req-08986d7b-a27e-40ea-a1c2-f0d9d98ab3b4'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:41.407895 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-08986d7b-a27e-40ea-a1c2-f0d9d98ab3b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db29ded4c53b41339cbc2415016de344', 'request_id': 'req-08986d7b-a27e-40ea-a1c2-f0d9d98ab3b4'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:41.426929 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-08986d7b-a27e-40ea-a1c2-f0d9d98ab3b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db29ded4c53b41339cbc2415016de344', 'request_id': 'req-08986d7b-a27e-40ea-a1c2-f0d9d98ab3b4'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:41.454101 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-08986d7b-a27e-40ea-a1c2-f0d9d98ab3b4 admin admin] The token cache is being invalidate because project db29ded4c53b41339cbc2415016de344 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:17:41.454593 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-08986d7b-a27e-40ea-a1c2-f0d9d98ab3b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:41.455789 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-08986d7b-a27e-40ea-a1c2-f0d9d98ab3b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:41.465519 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e552d0-974b-4a49-9543-99b789af2b39 service neutron] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:41.474788 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1091/2185] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:17:41 2022] DELETE /identity/v3/projects/db29ded4c53b41339cbc2415016de344 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:41.489386 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e552d0-974b-4a49-9543-99b789af2b39 service neutron] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:41.592032 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee9a2da-575a-4877-bee9-0689da4f5f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:41.592962 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee9a2da-575a-4877-bee9-0689da4f5f6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:41.593250 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee9a2da-575a-4877-bee9-0689da4f5f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:41.593534 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee9a2da-575a-4877-bee9-0689da4f5f6d admin admin] PATH_INFO: `/v3/users/ba8b741905ea47bdb29268a4d2e7b1b0` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:41.594874 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee9a2da-575a-4877-bee9-0689da4f5f6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba8b741905ea47bdb29268a4d2e7b1b0)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:41.596958 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee9a2da-575a-4877-bee9-0689da4f5f6d admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:41.605522 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:41.664861 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-4ee9a2da-575a-4877-bee9-0689da4f5f6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba8b741905ea47bdb29268a4d2e7b1b0'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:41.665654 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-4ee9a2da-575a-4877-bee9-0689da4f5f6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba8b741905ea47bdb29268a4d2e7b1b0'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:41.683923 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-4ee9a2da-575a-4877-bee9-0689da4f5f6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba8b741905ea47bdb29268a4d2e7b1b0'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:41.693352 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-4ee9a2da-575a-4877-bee9-0689da4f5f6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba8b741905ea47bdb29268a4d2e7b1b0'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:41.694224 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-4ee9a2da-575a-4877-bee9-0689da4f5f6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba8b741905ea47bdb29268a4d2e7b1b0'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:41.712601 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1095/2186] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:17:41 2022] DELETE /identity/v3/users/ba8b741905ea47bdb29268a4d2e7b1b0 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:42.153648 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08986d7b-a27e-40ea-a1c2-f0d9d98ab3b4 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:42.155496 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-08986d7b-a27e-40ea-a1c2-f0d9d98ab3b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:42.172231 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08986d7b-a27e-40ea-a1c2-f0d9d98ab3b4 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:42.250772 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec91c3-1e78-490d-84f6-cef648c2847c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:42.251694 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec91c3-1e78-490d-84f6-cef648c2847c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:42.251974 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec91c3-1e78-490d-84f6-cef648c2847c admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:42.252248 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec91c3-1e78-490d-84f6-cef648c2847c admin admin] PATH_INFO: `/v3/projects/0b01e452546e407fb37a564e36433c26` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:42.253573 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ec91c3-1e78-490d-84f6-cef648c2847c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b01e452546e407fb37a564e36433c26)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:42.255572 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ec91c3-1e78-490d-84f6-cef648c2847c admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:42.283239 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:42.313097 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-17ec91c3-1e78-490d-84f6-cef648c2847c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b01e452546e407fb37a564e36433c26', 'request_id': 'req-17ec91c3-1e78-490d-84f6-cef648c2847c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:42.321578 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-17ec91c3-1e78-490d-84f6-cef648c2847c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b01e452546e407fb37a564e36433c26', 'request_id': 'req-17ec91c3-1e78-490d-84f6-cef648c2847c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:42.331481 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-17ec91c3-1e78-490d-84f6-cef648c2847c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b01e452546e407fb37a564e36433c26', 'request_id': 'req-17ec91c3-1e78-490d-84f6-cef648c2847c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:42.345521 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-17ec91c3-1e78-490d-84f6-cef648c2847c admin admin] The token cache is being invalidate because project 0b01e452546e407fb37a564e36433c26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:17:42.345768 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-17ec91c3-1e78-490d-84f6-cef648c2847c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:42.346405 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-17ec91c3-1e78-490d-84f6-cef648c2847c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:42.348085 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1092/2187] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:17:42 2022] DELETE /identity/v3/projects/0b01e452546e407fb37a564e36433c26 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:42.354580 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0991161-29ed-445c-8323-29b19ea718e5 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:42.355053 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0991161-29ed-445c-8323-29b19ea718e5 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:42.355464 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0991161-29ed-445c-8323-29b19ea718e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:42.382415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-b0991161-29ed-445c-8323-29b19ea718e5 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:42.384213 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-b0991161-29ed-445c-8323-29b19ea718e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:42.483347 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1096/2188] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:42.496492 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880450fb-a3f1-46ec-aaa8-6ae755fbf940 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:42.496996 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880450fb-a3f1-46ec-aaa8-6ae755fbf940 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:42.497492 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880450fb-a3f1-46ec-aaa8-6ae755fbf940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:42.528273 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-880450fb-a3f1-46ec-aaa8-6ae755fbf940 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:42.530385 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-880450fb-a3f1-46ec-aaa8-6ae755fbf940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:42.544523 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1093/2189] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:42.549698 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0991161-29ed-445c-8323-29b19ea718e5 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:42.562204 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0991161-29ed-445c-8323-29b19ea718e5 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:42.567079 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb66a8-984f-465d-a9a8-0f33f947ccc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:42.567790 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb66a8-984f-465d-a9a8-0f33f947ccc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:42.568079 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb66a8-984f-465d-a9a8-0f33f947ccc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:42.568386 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb66a8-984f-465d-a9a8-0f33f947ccc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:42.569137 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bb66a8-984f-465d-a9a8-0f33f947ccc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:17:42.569444 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bb66a8-984f-465d-a9a8-0f33f947ccc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:42.571757 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bb66a8-984f-465d-a9a8-0f33f947ccc9 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:42.585587 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1097/2190] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:17:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:17:42.591347 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880450fb-a3f1-46ec-aaa8-6ae755fbf940 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:42.606133 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880450fb-a3f1-46ec-aaa8-6ae755fbf940 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:42.612910 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d476b-f325-4489-baac-340ad9030ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:42.614008 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d476b-f325-4489-baac-340ad9030ee1 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:42.614497 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d476b-f325-4489-baac-340ad9030ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:42.614970 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d476b-f325-4489-baac-340ad9030ee1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:42.616612 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d476b-f325-4489-baac-340ad9030ee1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:42.620924 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d476b-f325-4489-baac-340ad9030ee1 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:42.665107 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1094/2191] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:17:42 2022] POST /identity/v3/projects => generated 381 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:17:42.670999 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb66a8-984f-465d-a9a8-0f33f947ccc9 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:42.682927 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb66a8-984f-465d-a9a8-0f33f947ccc9 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:42.755465 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69726ed-6d0e-400e-af33-f96e024b310e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:42.756201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69726ed-6d0e-400e-af33-f96e024b310e admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:42.756471 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69726ed-6d0e-400e-af33-f96e024b310e admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:42.756906 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69726ed-6d0e-400e-af33-f96e024b310e admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:42.757632 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69726ed-6d0e-400e-af33-f96e024b310e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:42.759715 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69726ed-6d0e-400e-af33-f96e024b310e admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:42.778948 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:42.793138 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1098/2192] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:17:42 2022] POST /identity/v3/users => generated 291 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:17:42.797278 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d476b-f325-4489-baac-340ad9030ee1 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:42.813824 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d476b-f325-4489-baac-340ad9030ee1 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:42.818731 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bbbd02-d680-4e39-b040-01d16c67c48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:42.819418 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bbbd02-d680-4e39-b040-01d16c67c48d admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:42.819733 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bbbd02-d680-4e39-b040-01d16c67c48d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:42.820013 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bbbd02-d680-4e39-b040-01d16c67c48d admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:42.820842 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bbbd02-d680-4e39-b040-01d16c67c48d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:42.822475 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bbbd02-d680-4e39-b040-01d16c67c48d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:42.834036 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1095/2193] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:17:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:17:42.839034 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69726ed-6d0e-400e-af33-f96e024b310e admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:42.849587 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69726ed-6d0e-400e-af33-f96e024b310e admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:42.854974 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1928c3e7-df1f-4564-af5d-bcd8245aa364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:42.855695 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928c3e7-df1f-4564-af5d-bcd8245aa364 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:42.855984 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928c3e7-df1f-4564-af5d-bcd8245aa364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:42.856316 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928c3e7-df1f-4564-af5d-bcd8245aa364 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:42.857011 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1928c3e7-df1f-4564-af5d-bcd8245aa364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:42.858659 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1928c3e7-df1f-4564-af5d-bcd8245aa364 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:42.870106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1099/2194] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:17:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:17:42.876128 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bbbd02-d680-4e39-b040-01d16c67c48d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:42.887821 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bbbd02-d680-4e39-b040-01d16c67c48d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:42.893343 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ef82d-eca8-47a0-9550-d808f1a373f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:42.894389 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ef82d-eca8-47a0-9550-d808f1a373f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:42.894776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ef82d-eca8-47a0-9550-d808f1a373f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:42.895254 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ef82d-eca8-47a0-9550-d808f1a373f9 admin admin] PATH_INFO: `/v3/projects/f2df8e8a5cbf4bea879d37342914fb8c/users/c8f9cd77f8e7444fa43dd3bb00809028/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:42.914092 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884ef82d-eca8-47a0-9550-d808f1a373f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2df8e8a5cbf4bea879d37342914fb8c, user_id=c8f9cd77f8e7444fa43dd3bb00809028, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:42.917363 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884ef82d-eca8-47a0-9550-d808f1a373f9 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:42.933179 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1096/2195] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:17:42 2022] PUT /identity/v3/projects/f2df8e8a5cbf4bea879d37342914fb8c/users/c8f9cd77f8e7444fa43dd3bb00809028/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:42.944047 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73ac13-b7e0-49a2-ad21-9a5fde9a67e7 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:42.944340 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73ac13-b7e0-49a2-ad21-9a5fde9a67e7 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:42.944648 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73ac13-b7e0-49a2-ad21-9a5fde9a67e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:42.974827 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-5e73ac13-b7e0-49a2-ad21-9a5fde9a67e7 None None] MFA Rules not processed for user `c8f9cd77f8e7444fa43dd3bb00809028`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:42.976097 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-5e73ac13-b7e0-49a2-ad21-9a5fde9a67e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:43.076671 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1100/2196] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:43.083192 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ef82d-eca8-47a0-9550-d808f1a373f9 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:43.161952 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ef82d-eca8-47a0-9550-d808f1a373f9 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:43.174229 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980f5ca-be4d-4726-bd24-44d2a68a14bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:43.175053 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980f5ca-be4d-4726-bd24-44d2a68a14bd admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:43.175506 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980f5ca-be4d-4726-bd24-44d2a68a14bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:43.175874 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980f5ca-be4d-4726-bd24-44d2a68a14bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:43.176789 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2980f5ca-be4d-4726-bd24-44d2a68a14bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:43.180903 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2980f5ca-be4d-4726-bd24-44d2a68a14bd admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:43.225161 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1097/2197] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:17:43 2022] POST /identity/v3/projects => generated 381 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:17:43.232109 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e73ac13-b7e0-49a2-ad21-9a5fde9a67e7 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:43.255814 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e73ac13-b7e0-49a2-ad21-9a5fde9a67e7 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:43.411789 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dbe4c3-bded-45ed-ba20-f77424932c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:43.412895 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dbe4c3-bded-45ed-ba20-f77424932c46 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:43.413553 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dbe4c3-bded-45ed-ba20-f77424932c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:43.414062 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dbe4c3-bded-45ed-ba20-f77424932c46 admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:43.415412 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dbe4c3-bded-45ed-ba20-f77424932c46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:43.419344 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dbe4c3-bded-45ed-ba20-f77424932c46 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:43.456578 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1101/2198] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:17:43 2022] POST /identity/v3/users => generated 297 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:17:43.458548 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980f5ca-be4d-4726-bd24-44d2a68a14bd admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:43.474120 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980f5ca-be4d-4726-bd24-44d2a68a14bd admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:43.483406 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf65c425-1e3e-4051-98a7-aac0d932d4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:43.485278 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65c425-1e3e-4051-98a7-aac0d932d4ae admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:43.485735 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65c425-1e3e-4051-98a7-aac0d932d4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:43.486187 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65c425-1e3e-4051-98a7-aac0d932d4ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:43.488835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf65c425-1e3e-4051-98a7-aac0d932d4ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:43.490953 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf65c425-1e3e-4051-98a7-aac0d932d4ae admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:43.515525 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1098/2199] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:17:43 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:17:43.520655 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dbe4c3-bded-45ed-ba20-f77424932c46 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:43.541381 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dbe4c3-bded-45ed-ba20-f77424932c46 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:43.550844 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d39497-77d1-4ee8-b9f6-7d49f947045a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:43.552968 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d39497-77d1-4ee8-b9f6-7d49f947045a admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:43.553437 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d39497-77d1-4ee8-b9f6-7d49f947045a admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:43.555221 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d39497-77d1-4ee8-b9f6-7d49f947045a admin admin] PATH_INFO: `/v3/projects/57b501afd4454338893f923e1cc0573d/users/d8dd3c9c3ffa485b8914a771f4ffc880/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:43.581089 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d39497-77d1-4ee8-b9f6-7d49f947045a admin admin] RBAC: Authorizing `identity:create_grant(project_id=57b501afd4454338893f923e1cc0573d, user_id=d8dd3c9c3ffa485b8914a771f4ffc880, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:43.586997 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d39497-77d1-4ee8-b9f6-7d49f947045a admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:43.621322 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1102/2200] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:17:43 2022] PUT /identity/v3/projects/57b501afd4454338893f923e1cc0573d/users/d8dd3c9c3ffa485b8914a771f4ffc880/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:17:43.627770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf65c425-1e3e-4051-98a7-aac0d932d4ae admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:43.648003 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf65c425-1e3e-4051-98a7-aac0d932d4ae admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:43.682678 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:43.760659 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df36205b-0193-41ef-9fb0-62a88c9634d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:43.762744 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36205b-0193-41ef-9fb0-62a88c9634d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:43.763426 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36205b-0193-41ef-9fb0-62a88c9634d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:43.763954 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36205b-0193-41ef-9fb0-62a88c9634d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:43.765330 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df36205b-0193-41ef-9fb0-62a88c9634d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:43.767796 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df36205b-0193-41ef-9fb0-62a88c9634d4 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:43.782125 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1099/2201] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:17:43 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:17:43.788193 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d39497-77d1-4ee8-b9f6-7d49f947045a admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:43.801383 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d39497-77d1-4ee8-b9f6-7d49f947045a admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:43.807645 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced85dd7-be99-4a72-b520-368ca10406ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:43.808595 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced85dd7-be99-4a72-b520-368ca10406ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:43.808878 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced85dd7-be99-4a72-b520-368ca10406ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:43.809166 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced85dd7-be99-4a72-b520-368ca10406ea admin admin] PATH_INFO: `/v3/projects/57b501afd4454338893f923e1cc0573d/users/d8dd3c9c3ffa485b8914a771f4ffc880/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:43.811810 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced85dd7-be99-4a72-b520-368ca10406ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=57b501afd4454338893f923e1cc0573d, user_id=d8dd3c9c3ffa485b8914a771f4ffc880, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:43.827369 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced85dd7-be99-4a72-b520-368ca10406ea admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:43.856684 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1103/2202] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:17:43 2022] PUT /identity/v3/projects/57b501afd4454338893f923e1cc0573d/users/d8dd3c9c3ffa485b8914a771f4ffc880/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:43.875874 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca9ca4-c42a-4736-9e92-603395f7c939 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:43.876282 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca9ca4-c42a-4736-9e92-603395f7c939 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:43.876627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca9ca4-c42a-4736-9e92-603395f7c939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:43.918247 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-99ca9ca4-c42a-4736-9e92-603395f7c939 None None] MFA Rules not processed for user `d8dd3c9c3ffa485b8914a771f4ffc880`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:43.919669 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-99ca9ca4-c42a-4736-9e92-603395f7c939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:44.030931 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1100/2203] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:43 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:44.044971 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced85dd7-be99-4a72-b520-368ca10406ea admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:44.048712 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-ced85dd7-be99-4a72-b520-368ca10406ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:44.097097 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:44.140592 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced85dd7-be99-4a72-b520-368ca10406ea admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:44.146463 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc24531-41be-45cc-921b-0c51b89e1ab0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:44.147254 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc24531-41be-45cc-921b-0c51b89e1ab0 service cinder] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:44.147679 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc24531-41be-45cc-921b-0c51b89e1ab0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:44.148063 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc24531-41be-45cc-921b-0c51b89e1ab0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:44.159087 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc24531-41be-45cc-921b-0c51b89e1ab0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:44.160848 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc24531-41be-45cc-921b-0c51b89e1ab0 service cinder] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:44.174577 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1104/2204] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:17:44 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:17:44.300167 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ca9ca4-c42a-4736-9e92-603395f7c939 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:44.310705 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ca9ca4-c42a-4736-9e92-603395f7c939 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:44.314919 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b1a197-4da3-429d-a8c1-49f3ca7dda12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:44.315605 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b1a197-4da3-429d-a8c1-49f3ca7dda12 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:44.315893 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b1a197-4da3-429d-a8c1-49f3ca7dda12 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:44.316169 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b1a197-4da3-429d-a8c1-49f3ca7dda12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:44.385410 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b1a197-4da3-429d-a8c1-49f3ca7dda12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:44.386897 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b1a197-4da3-429d-a8c1-49f3ca7dda12 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:44.397195 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1101/2205] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:17:44 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:17:48.211099 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc24531-41be-45cc-921b-0c51b89e1ab0 service cinder] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:48.319943 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc24531-41be-45cc-921b-0c51b89e1ab0 service cinder] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:48.329337 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96f782-ae95-4a5d-ab98-d2abb9184124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:48.330245 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96f782-ae95-4a5d-ab98-d2abb9184124 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:48.330523 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96f782-ae95-4a5d-ab98-d2abb9184124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:48.330797 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96f782-ae95-4a5d-ab98-d2abb9184124 admin admin] PATH_INFO: `/v3/users/c8f9cd77f8e7444fa43dd3bb00809028` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:48.332150 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f96f782-ae95-4a5d-ab98-d2abb9184124 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8f9cd77f8e7444fa43dd3bb00809028)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:48.337161 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f96f782-ae95-4a5d-ab98-d2abb9184124 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:48.393325 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-3f96f782-ae95-4a5d-ab98-d2abb9184124 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8f9cd77f8e7444fa43dd3bb00809028'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:48.394008 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-3f96f782-ae95-4a5d-ab98-d2abb9184124 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8f9cd77f8e7444fa43dd3bb00809028'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:48.408514 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-3f96f782-ae95-4a5d-ab98-d2abb9184124 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8f9cd77f8e7444fa43dd3bb00809028'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:48.418091 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-3f96f782-ae95-4a5d-ab98-d2abb9184124 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8f9cd77f8e7444fa43dd3bb00809028'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:48.419079 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-3f96f782-ae95-4a5d-ab98-d2abb9184124 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8f9cd77f8e7444fa43dd3bb00809028'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:48.439122 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1105/2206] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:17:48 2022] DELETE /identity/v3/users/c8f9cd77f8e7444fa43dd3bb00809028 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:48.443956 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de3eef5-23f3-455f-9077-6dbdb7220cf7 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:48.444521 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de3eef5-23f3-455f-9077-6dbdb7220cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:48.444521 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de3eef5-23f3-455f-9077-6dbdb7220cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:48.464333 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-9de3eef5-23f3-455f-9077-6dbdb7220cf7 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:48.465579 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9de3eef5-23f3-455f-9077-6dbdb7220cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:48.503892 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:48.534256 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1102/2207] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:48.544722 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96f782-ae95-4a5d-ab98-d2abb9184124 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:48.546244 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-3f96f782-ae95-4a5d-ab98-d2abb9184124 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:48.615257 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96f782-ae95-4a5d-ab98-d2abb9184124 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:48.619171 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb96b8ca-0115-4265-8144-edea95e09c99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:48.619879 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96b8ca-0115-4265-8144-edea95e09c99 service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:48.620194 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96b8ca-0115-4265-8144-edea95e09c99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:48.620468 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96b8ca-0115-4265-8144-edea95e09c99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:48.628113 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:48.636117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb96b8ca-0115-4265-8144-edea95e09c99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:48.639616 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb96b8ca-0115-4265-8144-edea95e09c99 service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:48.658997 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1106/2208] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:17:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:17:49.035246 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de3eef5-23f3-455f-9077-6dbdb7220cf7 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:49.037432 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9de3eef5-23f3-455f-9077-6dbdb7220cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:49.064255 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de3eef5-23f3-455f-9077-6dbdb7220cf7 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:49.073461 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc43c7f0-04f9-4e13-b3cd-41a7d7164956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:49.075422 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43c7f0-04f9-4e13-b3cd-41a7d7164956 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:49.075978 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43c7f0-04f9-4e13-b3cd-41a7d7164956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:49.076479 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43c7f0-04f9-4e13-b3cd-41a7d7164956 admin admin] PATH_INFO: `/v3/projects/f2df8e8a5cbf4bea879d37342914fb8c` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:49.079642 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc43c7f0-04f9-4e13-b3cd-41a7d7164956 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2df8e8a5cbf4bea879d37342914fb8c)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:49.083833 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc43c7f0-04f9-4e13-b3cd-41a7d7164956 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:49.168354 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-cc43c7f0-04f9-4e13-b3cd-41a7d7164956 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2df8e8a5cbf4bea879d37342914fb8c', 'request_id': 'req-cc43c7f0-04f9-4e13-b3cd-41a7d7164956'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:49.177158 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-cc43c7f0-04f9-4e13-b3cd-41a7d7164956 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2df8e8a5cbf4bea879d37342914fb8c', 'request_id': 'req-cc43c7f0-04f9-4e13-b3cd-41a7d7164956'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:49.187851 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-cc43c7f0-04f9-4e13-b3cd-41a7d7164956 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2df8e8a5cbf4bea879d37342914fb8c', 'request_id': 'req-cc43c7f0-04f9-4e13-b3cd-41a7d7164956'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:49.203834 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-cc43c7f0-04f9-4e13-b3cd-41a7d7164956 admin admin] The token cache is being invalidate because project f2df8e8a5cbf4bea879d37342914fb8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:17:49.204294 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-cc43c7f0-04f9-4e13-b3cd-41a7d7164956 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:49.205172 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-cc43c7f0-04f9-4e13-b3cd-41a7d7164956 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:49.207124 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1103/2209] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:17:49 2022] DELETE /identity/v3/projects/f2df8e8a5cbf4bea879d37342914fb8c => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:49.212121 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb96b8ca-0115-4265-8144-edea95e09c99 service neutron] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:49.222755 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb96b8ca-0115-4265-8144-edea95e09c99 service neutron] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:49.316171 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c80e38-768a-47e9-be3d-8ab4137b81ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:49.318710 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c80e38-768a-47e9-be3d-8ab4137b81ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:49.319675 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c80e38-768a-47e9-be3d-8ab4137b81ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:49.321109 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c80e38-768a-47e9-be3d-8ab4137b81ec admin admin] PATH_INFO: `/v3/users/d8dd3c9c3ffa485b8914a771f4ffc880` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:49.324368 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c80e38-768a-47e9-be3d-8ab4137b81ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8dd3c9c3ffa485b8914a771f4ffc880)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:49.328811 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c80e38-768a-47e9-be3d-8ab4137b81ec admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:49.432071 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e9c80e38-768a-47e9-be3d-8ab4137b81ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8dd3c9c3ffa485b8914a771f4ffc880'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:49.432807 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e9c80e38-768a-47e9-be3d-8ab4137b81ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8dd3c9c3ffa485b8914a771f4ffc880'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:49.479645 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e9c80e38-768a-47e9-be3d-8ab4137b81ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8dd3c9c3ffa485b8914a771f4ffc880'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:49.491032 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e9c80e38-768a-47e9-be3d-8ab4137b81ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8dd3c9c3ffa485b8914a771f4ffc880'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:49.492417 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e9c80e38-768a-47e9-be3d-8ab4137b81ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8dd3c9c3ffa485b8914a771f4ffc880'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:49.518289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1107/2210] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:17:49 2022] DELETE /identity/v3/users/d8dd3c9c3ffa485b8914a771f4ffc880 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:49.944139 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc43c7f0-04f9-4e13-b3cd-41a7d7164956 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:49.945718 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-cc43c7f0-04f9-4e13-b3cd-41a7d7164956 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:49.962019 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc43c7f0-04f9-4e13-b3cd-41a7d7164956 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:50.043780 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f6e65-d2f0-4d08-901b-e1fb0e17cac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:50.044846 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f6e65-d2f0-4d08-901b-e1fb0e17cac1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:50.045006 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f6e65-d2f0-4d08-901b-e1fb0e17cac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:50.045254 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f6e65-d2f0-4d08-901b-e1fb0e17cac1 admin admin] PATH_INFO: `/v3/projects/57b501afd4454338893f923e1cc0573d` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:50.046786 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566f6e65-d2f0-4d08-901b-e1fb0e17cac1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57b501afd4454338893f923e1cc0573d)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:50.048797 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566f6e65-d2f0-4d08-901b-e1fb0e17cac1 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:50.114156 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-566f6e65-d2f0-4d08-901b-e1fb0e17cac1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57b501afd4454338893f923e1cc0573d', 'request_id': 'req-566f6e65-d2f0-4d08-901b-e1fb0e17cac1'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:50.122494 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-566f6e65-d2f0-4d08-901b-e1fb0e17cac1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57b501afd4454338893f923e1cc0573d', 'request_id': 'req-566f6e65-d2f0-4d08-901b-e1fb0e17cac1'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:50.132619 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-566f6e65-d2f0-4d08-901b-e1fb0e17cac1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57b501afd4454338893f923e1cc0573d', 'request_id': 'req-566f6e65-d2f0-4d08-901b-e1fb0e17cac1'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:50.137823 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:50.151350 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-566f6e65-d2f0-4d08-901b-e1fb0e17cac1 admin admin] The token cache is being invalidate because project 57b501afd4454338893f923e1cc0573d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:17:50.151944 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-566f6e65-d2f0-4d08-901b-e1fb0e17cac1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:50.153357 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-566f6e65-d2f0-4d08-901b-e1fb0e17cac1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:50.156104 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1104/2211] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:17:49 2022] DELETE /identity/v3/projects/57b501afd4454338893f923e1cc0573d => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:50.162504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79b2c6-2a1f-4a6b-90ca-03ae900dd832 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:50.162969 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79b2c6-2a1f-4a6b-90ca-03ae900dd832 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:50.163447 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79b2c6-2a1f-4a6b-90ca-03ae900dd832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:50.187386 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-ef79b2c6-2a1f-4a6b-90ca-03ae900dd832 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:50.189424 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-ef79b2c6-2a1f-4a6b-90ca-03ae900dd832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:50.261331 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1108/2212] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:50.272216 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23653dd-ebd5-411d-83ca-a8db0e02f818 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:50.272715 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23653dd-ebd5-411d-83ca-a8db0e02f818 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:50.273685 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23653dd-ebd5-411d-83ca-a8db0e02f818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:50.300208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-b23653dd-ebd5-411d-83ca-a8db0e02f818 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:50.301464 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b23653dd-ebd5-411d-83ca-a8db0e02f818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:50.310549 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1105/2213] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:50.315229 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79b2c6-2a1f-4a6b-90ca-03ae900dd832 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:50.325767 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79b2c6-2a1f-4a6b-90ca-03ae900dd832 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:50.330805 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6677dad7-8487-4685-b60f-bf929f0439c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:50.331490 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677dad7-8487-4685-b60f-bf929f0439c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:50.331799 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677dad7-8487-4685-b60f-bf929f0439c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:50.332073 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6677dad7-8487-4685-b60f-bf929f0439c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:50.332819 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6677dad7-8487-4685-b60f-bf929f0439c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:17:50.333124 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6677dad7-8487-4685-b60f-bf929f0439c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:50.334800 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6677dad7-8487-4685-b60f-bf929f0439c3 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:50.345840 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:50.349999 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1109/2214] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:17:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:17:50.355960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23653dd-ebd5-411d-83ca-a8db0e02f818 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:50.370680 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23653dd-ebd5-411d-83ca-a8db0e02f818 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:50.376198 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba8ebc-4438-4e62-90f0-cb1950f4388c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:50.376870 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba8ebc-4438-4e62-90f0-cb1950f4388c admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:50.377149 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba8ebc-4438-4e62-90f0-cb1950f4388c admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:50.377419 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba8ebc-4438-4e62-90f0-cb1950f4388c admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:50.378114 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ba8ebc-4438-4e62-90f0-cb1950f4388c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:50.380131 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ba8ebc-4438-4e62-90f0-cb1950f4388c admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:50.409842 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1106/2215] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:17:50 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:17:50.415207 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6677dad7-8487-4685-b60f-bf929f0439c3 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:50.426604 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6677dad7-8487-4685-b60f-bf929f0439c3 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:50.496583 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d92a1-e9fb-4d5f-b0fa-93961b264ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:50.497276 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d92a1-e9fb-4d5f-b0fa-93961b264ff2 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:50.497562 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d92a1-e9fb-4d5f-b0fa-93961b264ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:50.497839 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35d92a1-e9fb-4d5f-b0fa-93961b264ff2 admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:50.498560 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35d92a1-e9fb-4d5f-b0fa-93961b264ff2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:50.500738 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35d92a1-e9fb-4d5f-b0fa-93961b264ff2 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:50.528301 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1110/2216] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:17:50 2022] POST /identity/v3/users => generated 286 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:17:50.529708 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba8ebc-4438-4e62-90f0-cb1950f4388c admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:50.552086 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba8ebc-4438-4e62-90f0-cb1950f4388c admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:50.557206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd77bfb-9713-4dc2-9eb0-375e0812405c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:50.557933 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd77bfb-9713-4dc2-9eb0-375e0812405c admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:50.558215 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd77bfb-9713-4dc2-9eb0-375e0812405c admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:50.558558 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd77bfb-9713-4dc2-9eb0-375e0812405c admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:50.559460 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd77bfb-9713-4dc2-9eb0-375e0812405c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:50.561504 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd77bfb-9713-4dc2-9eb0-375e0812405c admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:50.576613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1107/2217] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:17:50.582344 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d92a1-e9fb-4d5f-b0fa-93961b264ff2 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:50.594687 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35d92a1-e9fb-4d5f-b0fa-93961b264ff2 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:50.600446 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3968b6e2-0acd-4e15-b5d6-843211235db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:50.601231 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3968b6e2-0acd-4e15-b5d6-843211235db9 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:50.601994 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3968b6e2-0acd-4e15-b5d6-843211235db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:50.602267 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3968b6e2-0acd-4e15-b5d6-843211235db9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:50.603011 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3968b6e2-0acd-4e15-b5d6-843211235db9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:50.604686 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3968b6e2-0acd-4e15-b5d6-843211235db9 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:50.620445 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1111/2218] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:17:50.625015 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd77bfb-9713-4dc2-9eb0-375e0812405c admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:50.636521 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd77bfb-9713-4dc2-9eb0-375e0812405c admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:50.641603 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8761ca8d-3812-415d-9ef0-173745f0150b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:50.642506 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761ca8d-3812-415d-9ef0-173745f0150b admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:50.642783 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761ca8d-3812-415d-9ef0-173745f0150b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:50.643058 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761ca8d-3812-415d-9ef0-173745f0150b admin admin] PATH_INFO: `/v3/projects/fb4bc70a605240dd8a16aa1e7d92eb46/users/c1351ffa5ba04e68aa944a4e74a852ef/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:50.662890 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8761ca8d-3812-415d-9ef0-173745f0150b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb4bc70a605240dd8a16aa1e7d92eb46, user_id=c1351ffa5ba04e68aa944a4e74a852ef, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:50.665700 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8761ca8d-3812-415d-9ef0-173745f0150b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:50.686652 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1108/2219] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:17:50 2022] PUT /identity/v3/projects/fb4bc70a605240dd8a16aa1e7d92eb46/users/c1351ffa5ba04e68aa944a4e74a852ef/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:50.699618 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031702f-1356-48de-b20e-07c2ebce8588 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:50.699891 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031702f-1356-48de-b20e-07c2ebce8588 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:50.700180 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031702f-1356-48de-b20e-07c2ebce8588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:50.737605 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-a031702f-1356-48de-b20e-07c2ebce8588 None None] MFA Rules not processed for user `c1351ffa5ba04e68aa944a4e74a852ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:50.738880 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-a031702f-1356-48de-b20e-07c2ebce8588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:50.808852 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55154d47-3f55-4220-bb5f-69dd0db7175a None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:50.809298 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55154d47-3f55-4220-bb5f-69dd0db7175a None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:50.809709 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55154d47-3f55-4220-bb5f-69dd0db7175a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:50.843369 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-55154d47-3f55-4220-bb5f-69dd0db7175a None None] MFA Rules not processed for user `a32702a67a7b49ffa153fc59251fd169`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:50.850875 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1112/2220] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:50.852476 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-55154d47-3f55-4220-bb5f-69dd0db7175a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:50.861579 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a031702f-1356-48de-b20e-07c2ebce8588 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:50.863803 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-a031702f-1356-48de-b20e-07c2ebce8588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:50.952628 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a031702f-1356-48de-b20e-07c2ebce8588 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:50.959289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a8611-7cf4-4bc3-befc-e69f0d585e3d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ce9e6497fdb4068b23aa3f7b2c19536', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:50.960375 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a8611-7cf4-4bc3-befc-e69f0d585e3d service glance] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:50.961610 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a8611-7cf4-4bc3-befc-e69f0d585e3d service glance] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:50.962098 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a8611-7cf4-4bc3-befc-e69f0d585e3d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:50.965957 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1109/2221] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Jul 13 23:17:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:50.981122 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224a8611-7cf4-4bc3-befc-e69f0d585e3d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:50.984606 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224a8611-7cf4-4bc3-befc-e69f0d585e3d service glance] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:51.002089 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1113/2222] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:17:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:17:51.034191 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55154d47-3f55-4220-bb5f-69dd0db7175a None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:51.036035 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-55154d47-3f55-4220-bb5f-69dd0db7175a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:51.123186 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55154d47-3f55-4220-bb5f-69dd0db7175a None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:51.133848 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b85d2f-9af8-45ec-9c35-801f2b4e5e0c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786fc44611804d3ca6280ffcab415569', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:51.134558 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b85d2f-9af8-45ec-9c35-801f2b4e5e0c service swift] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:51.135004 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b85d2f-9af8-45ec-9c35-801f2b4e5e0c service swift] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:51.135291 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b85d2f-9af8-45ec-9c35-801f2b4e5e0c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:51.144781 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b85d2f-9af8-45ec-9c35-801f2b4e5e0c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:51.146438 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b85d2f-9af8-45ec-9c35-801f2b4e5e0c service swift] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:51.156832 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1110/2223] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Jul 13 23:17:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 13 23:17:51.995162 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ffe66-25a8-401f-9e07-be6888d1424f None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:51.996085 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ffe66-25a8-401f-9e07-be6888d1424f None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:51.996085 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ffe66-25a8-401f-9e07-be6888d1424f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:52.018413 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-bb7ffe66-25a8-401f-9e07-be6888d1424f None None] MFA Rules not processed for user `a32702a67a7b49ffa153fc59251fd169`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:52.020225 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-bb7ffe66-25a8-401f-9e07-be6888d1424f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:52.031555 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1114/2224] 10.222.0.60 () {58 vars in 967 bytes} [Wed Jul 13 23:17:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:52.053712 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b85d2f-9af8-45ec-9c35-801f2b4e5e0c service swift] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:52.074286 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b85d2f-9af8-45ec-9c35-801f2b4e5e0c service swift] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:52.083998 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0600b086-d038-4784-bc36-798cc916243e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786fc44611804d3ca6280ffcab415569', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:52.085164 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600b086-d038-4784-bc36-798cc916243e service swift] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:52.086287 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600b086-d038-4784-bc36-798cc916243e service swift] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:52.086792 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600b086-d038-4784-bc36-798cc916243e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:52.103960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0600b086-d038-4784-bc36-798cc916243e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:52.106823 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0600b086-d038-4784-bc36-798cc916243e service swift] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:52.140162 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1111/2225] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Jul 13 23:17:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 23:17:52.408570 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d92dc-bc64-48a2-b41f-6d6318b6763a None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:52.409074 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d92dc-bc64-48a2-b41f-6d6318b6763a None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:52.409582 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d92dc-bc64-48a2-b41f-6d6318b6763a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:52.421994 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:52.442383 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-003d92dc-bc64-48a2-b41f-6d6318b6763a None None] MFA Rules not processed for user `a32702a67a7b49ffa153fc59251fd169`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:52.444567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-003d92dc-bc64-48a2-b41f-6d6318b6763a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:52.457829 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1115/2226] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Jul 13 23:17:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:52.471437 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0600b086-d038-4784-bc36-798cc916243e service swift] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:52.482672 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0600b086-d038-4784-bc36-798cc916243e service swift] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:52.487926 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e6636f-825a-4611-abd2-eb6519d99990 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786fc44611804d3ca6280ffcab415569', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:52.488584 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6636f-825a-4611-abd2-eb6519d99990 service swift] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:52.488868 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6636f-825a-4611-abd2-eb6519d99990 service swift] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:52.489168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6636f-825a-4611-abd2-eb6519d99990 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:52.499135 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e6636f-825a-4611-abd2-eb6519d99990 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:52.500856 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e6636f-825a-4611-abd2-eb6519d99990 service swift] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:52.505053 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:52.513731 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1112/2227] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Jul 13 23:17:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 13 23:17:52.903613 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3b4b0-8b46-41be-9716-ac31f78fcf6f None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:52.903901 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3b4b0-8b46-41be-9716-ac31f78fcf6f None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:52.904174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3b4b0-8b46-41be-9716-ac31f78fcf6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:52.924890 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-17d3b4b0-8b46-41be-9716-ac31f78fcf6f None None] MFA Rules not processed for user `a32702a67a7b49ffa153fc59251fd169`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:52.926152 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-17d3b4b0-8b46-41be-9716-ac31f78fcf6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:52.935440 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1116/2228] 10.222.0.60 () {58 vars in 967 bytes} [Wed Jul 13 23:17:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:52.948788 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e6636f-825a-4611-abd2-eb6519d99990 service swift] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:52.961312 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e6636f-825a-4611-abd2-eb6519d99990 service swift] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:52.967073 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1bc3dc-b475-4fe5-949b-39da834b8097 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786fc44611804d3ca6280ffcab415569', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:52.967878 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1bc3dc-b475-4fe5-949b-39da834b8097 service swift] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:52.968205 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1bc3dc-b475-4fe5-949b-39da834b8097 service swift] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:52.968531 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1bc3dc-b475-4fe5-949b-39da834b8097 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:52.980767 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1bc3dc-b475-4fe5-949b-39da834b8097 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:52.982440 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1bc3dc-b475-4fe5-949b-39da834b8097 service swift] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:52.992897 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1113/2229] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Jul 13 23:17:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 23:17:55.423865 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d3b4b0-8b46-41be-9716-ac31f78fcf6f None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:55.425705 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-17d3b4b0-8b46-41be-9716-ac31f78fcf6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:55.508574 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d3b4b0-8b46-41be-9716-ac31f78fcf6f None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:55.513918 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4969024-8ca7-4835-b5bc-0b280e69c3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:55.514883 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4969024-8ca7-4835-b5bc-0b280e69c3bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:55.515212 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4969024-8ca7-4835-b5bc-0b280e69c3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:55.515571 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4969024-8ca7-4835-b5bc-0b280e69c3bc admin admin] PATH_INFO: `/v3/users/138ec380578e47599b5a78031a412071` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:55.517246 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4969024-8ca7-4835-b5bc-0b280e69c3bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=138ec380578e47599b5a78031a412071)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:55.519572 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4969024-8ca7-4835-b5bc-0b280e69c3bc admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:55.582520 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-b4969024-8ca7-4835-b5bc-0b280e69c3bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '138ec380578e47599b5a78031a412071'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:55.583368 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-b4969024-8ca7-4835-b5bc-0b280e69c3bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '138ec380578e47599b5a78031a412071'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:55.600013 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-b4969024-8ca7-4835-b5bc-0b280e69c3bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '138ec380578e47599b5a78031a412071'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:55.609842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-b4969024-8ca7-4835-b5bc-0b280e69c3bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '138ec380578e47599b5a78031a412071'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:55.610614 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-b4969024-8ca7-4835-b5bc-0b280e69c3bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '138ec380578e47599b5a78031a412071'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:55.627461 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1117/2230] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:17:55 2022] DELETE /identity/v3/users/138ec380578e47599b5a78031a412071 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:17:55.634881 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed152e22-d7d7-445c-86f6-e6a5b379ef48 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:55.634881 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed152e22-d7d7-445c-86f6-e6a5b379ef48 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:55.635589 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed152e22-d7d7-445c-86f6-e6a5b379ef48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:55.655271 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-ed152e22-d7d7-445c-86f6-e6a5b379ef48 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:55.656565 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ed152e22-d7d7-445c-86f6-e6a5b379ef48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:55.729898 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1114/2231] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:17:55.742753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4969024-8ca7-4835-b5bc-0b280e69c3bc admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:55.744635 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-b4969024-8ca7-4835-b5bc-0b280e69c3bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:55.848912 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4969024-8ca7-4835-b5bc-0b280e69c3bc admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:55.852885 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5825875c-992c-42b6-96ce-12216161e693 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:55.853744 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825875c-992c-42b6-96ce-12216161e693 service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:55.854067 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825875c-992c-42b6-96ce-12216161e693 service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:55.854414 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825875c-992c-42b6-96ce-12216161e693 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:55.866434 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5825875c-992c-42b6-96ce-12216161e693 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:55.868625 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5825875c-992c-42b6-96ce-12216161e693 service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:55.880947 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1118/2232] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:17:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:17:56.144226 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed152e22-d7d7-445c-86f6-e6a5b379ef48 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:56.146700 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ed152e22-d7d7-445c-86f6-e6a5b379ef48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:56.169162 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed152e22-d7d7-445c-86f6-e6a5b379ef48 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:56.175025 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344c3eb4-2f14-4513-976e-4aaa6f4a358e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:56.176192 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c3eb4-2f14-4513-976e-4aaa6f4a358e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:56.176581 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c3eb4-2f14-4513-976e-4aaa6f4a358e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:56.177001 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c3eb4-2f14-4513-976e-4aaa6f4a358e admin admin] PATH_INFO: `/v3/projects/68ad7a3acf6d4636a67341c059d66568` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:56.178861 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344c3eb4-2f14-4513-976e-4aaa6f4a358e admin admin] RBAC: Authorizing `identity:delete_project(project_id=68ad7a3acf6d4636a67341c059d66568)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:56.181989 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344c3eb4-2f14-4513-976e-4aaa6f4a358e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:56.255519 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-344c3eb4-2f14-4513-976e-4aaa6f4a358e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68ad7a3acf6d4636a67341c059d66568', 'request_id': 'req-344c3eb4-2f14-4513-976e-4aaa6f4a358e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:56.268037 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-344c3eb4-2f14-4513-976e-4aaa6f4a358e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68ad7a3acf6d4636a67341c059d66568', 'request_id': 'req-344c3eb4-2f14-4513-976e-4aaa6f4a358e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:56.281020 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-344c3eb4-2f14-4513-976e-4aaa6f4a358e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68ad7a3acf6d4636a67341c059d66568', 'request_id': 'req-344c3eb4-2f14-4513-976e-4aaa6f4a358e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:56.294814 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-344c3eb4-2f14-4513-976e-4aaa6f4a358e admin admin] The token cache is being invalidate because project 68ad7a3acf6d4636a67341c059d66568 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:17:56.295195 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-344c3eb4-2f14-4513-976e-4aaa6f4a358e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:56.296145 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-344c3eb4-2f14-4513-976e-4aaa6f4a358e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:17:56.297914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1115/2233] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:17:56 2022] DELETE /identity/v3/projects/68ad7a3acf6d4636a67341c059d66568 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:56.304749 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73840141-96a9-4296-9ffb-3fd5805a9b4c None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:56.305240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73840141-96a9-4296-9ffb-3fd5805a9b4c None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:56.305704 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73840141-96a9-4296-9ffb-3fd5805a9b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:56.324617 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-73840141-96a9-4296-9ffb-3fd5805a9b4c None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:56.327136 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-73840141-96a9-4296-9ffb-3fd5805a9b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:56.338392 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:56.423903 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1119/2234] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:17:56.492427 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9dafc2-3bc6-40a9-abe3-bb5dd67c8260 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:56.492746 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9dafc2-3bc6-40a9-abe3-bb5dd67c8260 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:56.493033 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9dafc2-3bc6-40a9-abe3-bb5dd67c8260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:56.517307 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-7c9dafc2-3bc6-40a9-abe3-bb5dd67c8260 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:56.518560 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7c9dafc2-3bc6-40a9-abe3-bb5dd67c8260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:56.531015 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1116/2235] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:17:56.536006 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73840141-96a9-4296-9ffb-3fd5805a9b4c None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:56.550627 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73840141-96a9-4296-9ffb-3fd5805a9b4c None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:56.558128 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dbe0e2-416b-41de-8f7e-6ce6b9b58544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:56.558814 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dbe0e2-416b-41de-8f7e-6ce6b9b58544 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:56.559123 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dbe0e2-416b-41de-8f7e-6ce6b9b58544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:56.559408 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dbe0e2-416b-41de-8f7e-6ce6b9b58544 admin admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:56.560221 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dbe0e2-416b-41de-8f7e-6ce6b9b58544 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:17:56.560529 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dbe0e2-416b-41de-8f7e-6ce6b9b58544 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:56.562253 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dbe0e2-416b-41de-8f7e-6ce6b9b58544 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:56.576519 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1120/2236] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:17:56.583187 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9dafc2-3bc6-40a9-abe3-bb5dd67c8260 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:56.594241 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9dafc2-3bc6-40a9-abe3-bb5dd67c8260 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:56.602690 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90555400-a179-4c55-b4c4-1df92180749b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:56.603901 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90555400-a179-4c55-b4c4-1df92180749b admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:56.604408 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90555400-a179-4c55-b4c4-1df92180749b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:56.604905 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90555400-a179-4c55-b4c4-1df92180749b admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:56.606269 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90555400-a179-4c55-b4c4-1df92180749b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:56.610132 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90555400-a179-4c55-b4c4-1df92180749b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:56.654861 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1117/2237] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:17:56 2022] POST /identity/v3/projects => generated 389 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:17:56.659725 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dbe0e2-416b-41de-8f7e-6ce6b9b58544 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:56.670512 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dbe0e2-416b-41de-8f7e-6ce6b9b58544 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:56.744444 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be04172-bcbb-4d55-829f-9283be68e647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:56.745473 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be04172-bcbb-4d55-829f-9283be68e647 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:56.745980 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be04172-bcbb-4d55-829f-9283be68e647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:56.746484 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be04172-bcbb-4d55-829f-9283be68e647 admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:56.747845 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be04172-bcbb-4d55-829f-9283be68e647 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:56.751340 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be04172-bcbb-4d55-829f-9283be68e647 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:56.778899 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1121/2238] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:17:56 2022] POST /identity/v3/users => generated 295 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:17:56.784904 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90555400-a179-4c55-b4c4-1df92180749b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:56.796042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90555400-a179-4c55-b4c4-1df92180749b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:56.801290 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d7dd88-86d1-421f-a495-d56d3537f588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:56.802088 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7dd88-86d1-421f-a495-d56d3537f588 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:56.802734 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7dd88-86d1-421f-a495-d56d3537f588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:56.803107 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7dd88-86d1-421f-a495-d56d3537f588 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:56.804077 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d7dd88-86d1-421f-a495-d56d3537f588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:56.805929 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d7dd88-86d1-421f-a495-d56d3537f588 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:56.818030 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1118/2239] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:17:56.825083 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be04172-bcbb-4d55-829f-9283be68e647 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:56.844731 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be04172-bcbb-4d55-829f-9283be68e647 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:56.852847 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8563ddf-0176-4496-84a3-3d559be573e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:56.853966 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8563ddf-0176-4496-84a3-3d559be573e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:56.854437 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8563ddf-0176-4496-84a3-3d559be573e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:56.854919 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8563ddf-0176-4496-84a3-3d559be573e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:56.856311 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8563ddf-0176-4496-84a3-3d559be573e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:56.859163 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8563ddf-0176-4496-84a3-3d559be573e8 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:56.876953 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1122/2240] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:17:56.881187 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d7dd88-86d1-421f-a495-d56d3537f588 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:56.893216 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d7dd88-86d1-421f-a495-d56d3537f588 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:56.898935 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5393a28a-d700-4829-90ab-093582be225a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:56.899861 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393a28a-d700-4829-90ab-093582be225a admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:56.900159 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393a28a-d700-4829-90ab-093582be225a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:56.900410 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393a28a-d700-4829-90ab-093582be225a admin admin] PATH_INFO: `/v3/projects/dde398a3d3514b2faacff0885236b933/users/f301622e9a684ad98446636b6768e8b8/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:56.916973 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5393a28a-d700-4829-90ab-093582be225a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dde398a3d3514b2faacff0885236b933, user_id=f301622e9a684ad98446636b6768e8b8, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:56.919679 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5393a28a-d700-4829-90ab-093582be225a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:56.935119 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1119/2241] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:17:56 2022] PUT /identity/v3/projects/dde398a3d3514b2faacff0885236b933/users/f301622e9a684ad98446636b6768e8b8/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:17:56.946400 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12ea0d-356e-4a4e-8edb-54d2b72ef440 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:56.946689 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12ea0d-356e-4a4e-8edb-54d2b72ef440 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:56.946946 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12ea0d-356e-4a4e-8edb-54d2b72ef440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:56.980074 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-1b12ea0d-356e-4a4e-8edb-54d2b72ef440 None None] MFA Rules not processed for user `f301622e9a684ad98446636b6768e8b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:56.981351 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-1b12ea0d-356e-4a4e-8edb-54d2b72ef440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:57.121061 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1123/2242] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:17:57.151579 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5393a28a-d700-4829-90ab-093582be225a admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:17:57.154423 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5393a28a-d700-4829-90ab-093582be225a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:57.171104 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:57.262543 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5393a28a-d700-4829-90ab-093582be225a admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:17:57.270205 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17d95f-6d32-4617-bd49-45f3961cde39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:17:57.272134 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17d95f-6d32-4617-bd49-45f3961cde39 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:57.272649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17d95f-6d32-4617-bd49-45f3961cde39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:57.273143 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17d95f-6d32-4617-bd49-45f3961cde39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:57.289767 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a17d95f-6d32-4617-bd49-45f3961cde39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:17:57.292696 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a17d95f-6d32-4617-bd49-45f3961cde39 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:17:57.310764 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1120/2243] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:17:57 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:17:58.183908 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff34dc9-f1d6-47cb-a65b-642e006b5c99 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:17:58.184747 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff34dc9-f1d6-47cb-a65b-642e006b5c99 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:17:58.184747 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff34dc9-f1d6-47cb-a65b-642e006b5c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:17:58.214332 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-eff34dc9-f1d6-47cb-a65b-642e006b5c99 None None] MFA Rules not processed for user `c3871ceed47e4fcea07c5f879c8462cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:17:58.216619 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-eff34dc9-f1d6-47cb-a65b-642e006b5c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:17:58.224530 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:17:58.326853 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1124/2244] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:17:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:18:07.302434 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=24 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:18:08.313592 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=21 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:18:22.469368 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17d95f-6d32-4617-bd49-45f3961cde39 service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:18:22.471668 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3a17d95f-6d32-4617-bd49-45f3961cde39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:18:22.611126 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17d95f-6d32-4617-bd49-45f3961cde39 service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:18:22.623871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605de307-581b-4204-bf63-d88507c369b1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64824ba84a024930bbc177f8a8d9a046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:18:22.624999 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605de307-581b-4204-bf63-d88507c369b1 service placement] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:18:22.625508 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605de307-581b-4204-bf63-d88507c369b1 service placement] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:18:22.626717 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605de307-581b-4204-bf63-d88507c369b1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:18:22.632799 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-605de307-581b-4204-bf63-d88507c369b1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:18:22.659631 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605de307-581b-4204-bf63-d88507c369b1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:18:22.663771 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605de307-581b-4204-bf63-d88507c369b1 service placement] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:18:22.698168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1121/2245] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:18:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:18:25.476514 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50727052-4802-4efa-8cfa-0ea4c830befb None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:18:25.477504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50727052-4802-4efa-8cfa-0ea4c830befb None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:18:25.477504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50727052-4802-4efa-8cfa-0ea4c830befb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:18:25.499823 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-50727052-4802-4efa-8cfa-0ea4c830befb None None] MFA Rules not processed for user `e2e2bd6cf0a44d529ef9b2678c467aba`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:18:25.501286 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-50727052-4802-4efa-8cfa-0ea4c830befb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:18:25.565444 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1125/2246] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:18:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:18:28.039813 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605de307-581b-4204-bf63-d88507c369b1 service placement] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:18:28.185367 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605de307-581b-4204-bf63-d88507c369b1 service placement] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:18:28.191408 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999a4d3-22d6-4081-bd3e-9b374a54b310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:18:28.192418 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999a4d3-22d6-4081-bd3e-9b374a54b310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:18:28.192692 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999a4d3-22d6-4081-bd3e-9b374a54b310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:18:28.193657 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999a4d3-22d6-4081-bd3e-9b374a54b310 admin admin] PATH_INFO: `/v3/users/f301622e9a684ad98446636b6768e8b8` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:18:28.195564 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9999a4d3-22d6-4081-bd3e-9b374a54b310 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f301622e9a684ad98446636b6768e8b8)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:18:28.198740 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9999a4d3-22d6-4081-bd3e-9b374a54b310 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:18:28.295377 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-9999a4d3-22d6-4081-bd3e-9b374a54b310 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f301622e9a684ad98446636b6768e8b8'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:18:28.296818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-9999a4d3-22d6-4081-bd3e-9b374a54b310 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f301622e9a684ad98446636b6768e8b8'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:18:28.325763 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-9999a4d3-22d6-4081-bd3e-9b374a54b310 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f301622e9a684ad98446636b6768e8b8'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:18:28.339512 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-9999a4d3-22d6-4081-bd3e-9b374a54b310 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f301622e9a684ad98446636b6768e8b8'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:18:28.350782 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-9999a4d3-22d6-4081-bd3e-9b374a54b310 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f301622e9a684ad98446636b6768e8b8'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:18:28.366537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1122/2247] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:18:28 2022] DELETE /identity/v3/users/f301622e9a684ad98446636b6768e8b8 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:18:28.374707 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89dd00e-f491-4a5e-a51d-daed8014041e None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:18:28.375613 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89dd00e-f491-4a5e-a51d-daed8014041e None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:18:28.376290 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89dd00e-f491-4a5e-a51d-daed8014041e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:18:28.419062 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-f89dd00e-f491-4a5e-a51d-daed8014041e None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:18:28.422256 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-f89dd00e-f491-4a5e-a51d-daed8014041e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:18:28.524712 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1126/2248] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:18:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:18:28.536134 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999a4d3-22d6-4081-bd3e-9b374a54b310 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:18:28.539057 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9999a4d3-22d6-4081-bd3e-9b374a54b310 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:18:28.554781 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:18:28.619242 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999a4d3-22d6-4081-bd3e-9b374a54b310 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:18:28.624558 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e7a9ad-54c8-4f7b-bb9c-caa14aa48d97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:18:28.625308 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e7a9ad-54c8-4f7b-bb9c-caa14aa48d97 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:18:28.625728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e7a9ad-54c8-4f7b-bb9c-caa14aa48d97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:18:28.626228 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e7a9ad-54c8-4f7b-bb9c-caa14aa48d97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:18:28.643669 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e7a9ad-54c8-4f7b-bb9c-caa14aa48d97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:18:28.647073 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e7a9ad-54c8-4f7b-bb9c-caa14aa48d97 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:18:28.666259 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1123/2249] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:18:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:18:29.358054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89dd00e-f491-4a5e-a51d-daed8014041e None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:18:29.360317 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-f89dd00e-f491-4a5e-a51d-daed8014041e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:18:29.382360 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89dd00e-f491-4a5e-a51d-daed8014041e None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:18:29.399582 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf40a5ab-7678-48b7-9a64-ec86fd1cb753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:18:29.401246 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40a5ab-7678-48b7-9a64-ec86fd1cb753 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:18:29.401664 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40a5ab-7678-48b7-9a64-ec86fd1cb753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:18:29.402187 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf40a5ab-7678-48b7-9a64-ec86fd1cb753 admin admin] PATH_INFO: `/v3/projects/dde398a3d3514b2faacff0885236b933` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:18:29.404589 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf40a5ab-7678-48b7-9a64-ec86fd1cb753 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dde398a3d3514b2faacff0885236b933)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:18:29.408831 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf40a5ab-7678-48b7-9a64-ec86fd1cb753 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:18:29.489062 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-cf40a5ab-7678-48b7-9a64-ec86fd1cb753 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dde398a3d3514b2faacff0885236b933', 'request_id': 'req-cf40a5ab-7678-48b7-9a64-ec86fd1cb753'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:18:29.498108 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-cf40a5ab-7678-48b7-9a64-ec86fd1cb753 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dde398a3d3514b2faacff0885236b933', 'request_id': 'req-cf40a5ab-7678-48b7-9a64-ec86fd1cb753'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:18:29.509931 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-cf40a5ab-7678-48b7-9a64-ec86fd1cb753 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dde398a3d3514b2faacff0885236b933', 'request_id': 'req-cf40a5ab-7678-48b7-9a64-ec86fd1cb753'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:18:29.530504 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-cf40a5ab-7678-48b7-9a64-ec86fd1cb753 admin admin] The token cache is being invalidate because project dde398a3d3514b2faacff0885236b933 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:18:29.531028 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-cf40a5ab-7678-48b7-9a64-ec86fd1cb753 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:18:29.532270 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-cf40a5ab-7678-48b7-9a64-ec86fd1cb753 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:18:29.544581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1127/2250] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:18:29 2022] DELETE /identity/v3/projects/dde398a3d3514b2faacff0885236b933 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:18:38.656936 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=22 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:18:39.523602 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:18:45.629361 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e7a9ad-54c8-4f7b-bb9c-caa14aa48d97 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:18:45.631795 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-21e7a9ad-54c8-4f7b-bb9c-caa14aa48d97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:18:45.644954 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e7a9ad-54c8-4f7b-bb9c-caa14aa48d97 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:18:45.708307 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8582ce60-ec39-4bf1-823f-fc345f9075dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:18:45.710416 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582ce60-ec39-4bf1-823f-fc345f9075dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:18:45.710708 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582ce60-ec39-4bf1-823f-fc345f9075dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:18:45.711868 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582ce60-ec39-4bf1-823f-fc345f9075dc admin admin] PATH_INFO: `/v3/users/c1351ffa5ba04e68aa944a4e74a852ef` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:18:45.712541 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8582ce60-ec39-4bf1-823f-fc345f9075dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1351ffa5ba04e68aa944a4e74a852ef)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:18:45.714532 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8582ce60-ec39-4bf1-823f-fc345f9075dc admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:18:45.774695 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8582ce60-ec39-4bf1-823f-fc345f9075dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1351ffa5ba04e68aa944a4e74a852ef'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:18:45.775866 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8582ce60-ec39-4bf1-823f-fc345f9075dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1351ffa5ba04e68aa944a4e74a852ef'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:18:45.797115 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8582ce60-ec39-4bf1-823f-fc345f9075dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1351ffa5ba04e68aa944a4e74a852ef'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:18:45.811447 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8582ce60-ec39-4bf1-823f-fc345f9075dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1351ffa5ba04e68aa944a4e74a852ef'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:18:45.824289 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8582ce60-ec39-4bf1-823f-fc345f9075dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1351ffa5ba04e68aa944a4e74a852ef'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:18:45.835776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1124/2251] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:18:45 2022] DELETE /identity/v3/users/c1351ffa5ba04e68aa944a4e74a852ef => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:18:45.841400 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be7e85-b63b-434d-8a9d-d1f97410ca84 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:18:45.841916 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be7e85-b63b-434d-8a9d-d1f97410ca84 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:18:45.842315 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be7e85-b63b-434d-8a9d-d1f97410ca84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:18:45.869178 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-18be7e85-b63b-434d-8a9d-d1f97410ca84 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:18:45.871194 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-18be7e85-b63b-434d-8a9d-d1f97410ca84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:18:45.955027 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1128/2252] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:18:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:18:45.966042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8582ce60-ec39-4bf1-823f-fc345f9075dc admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:18:45.968159 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8582ce60-ec39-4bf1-823f-fc345f9075dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:18:46.036489 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8582ce60-ec39-4bf1-823f-fc345f9075dc admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:18:46.040652 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09f7433-7a11-4a50-8fe9-d58eeac2e2c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:18:46.041318 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f7433-7a11-4a50-8fe9-d58eeac2e2c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:18:46.041747 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f7433-7a11-4a50-8fe9-d58eeac2e2c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:18:46.042028 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f7433-7a11-4a50-8fe9-d58eeac2e2c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:18:46.053880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09f7433-7a11-4a50-8fe9-d58eeac2e2c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:18:46.055614 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09f7433-7a11-4a50-8fe9-d58eeac2e2c4 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:18:46.067902 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1125/2253] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:18:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:18:46.413332 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18be7e85-b63b-434d-8a9d-d1f97410ca84 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:18:46.415134 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-18be7e85-b63b-434d-8a9d-d1f97410ca84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:18:46.438516 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18be7e85-b63b-434d-8a9d-d1f97410ca84 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:18:46.447976 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde91d21-5a29-4a9a-830c-e2c9d9874f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:18:46.449937 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde91d21-5a29-4a9a-830c-e2c9d9874f28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:18:46.450808 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde91d21-5a29-4a9a-830c-e2c9d9874f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:18:46.451407 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde91d21-5a29-4a9a-830c-e2c9d9874f28 admin admin] PATH_INFO: `/v3/projects/fb4bc70a605240dd8a16aa1e7d92eb46` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:18:46.454293 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde91d21-5a29-4a9a-830c-e2c9d9874f28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb4bc70a605240dd8a16aa1e7d92eb46)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:18:46.458739 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde91d21-5a29-4a9a-830c-e2c9d9874f28 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:18:46.537888 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-dde91d21-5a29-4a9a-830c-e2c9d9874f28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb4bc70a605240dd8a16aa1e7d92eb46', 'request_id': 'req-dde91d21-5a29-4a9a-830c-e2c9d9874f28'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:18:46.545328 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-dde91d21-5a29-4a9a-830c-e2c9d9874f28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb4bc70a605240dd8a16aa1e7d92eb46', 'request_id': 'req-dde91d21-5a29-4a9a-830c-e2c9d9874f28'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:18:46.554618 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-dde91d21-5a29-4a9a-830c-e2c9d9874f28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb4bc70a605240dd8a16aa1e7d92eb46', 'request_id': 'req-dde91d21-5a29-4a9a-830c-e2c9d9874f28'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:18:46.567371 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-dde91d21-5a29-4a9a-830c-e2c9d9874f28 admin admin] The token cache is being invalidate because project fb4bc70a605240dd8a16aa1e7d92eb46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:18:46.567694 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-dde91d21-5a29-4a9a-830c-e2c9d9874f28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:18:46.568538 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-dde91d21-5a29-4a9a-830c-e2c9d9874f28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:18:46.570158 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1129/2254] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:18:46 2022] DELETE /identity/v3/projects/fb4bc70a605240dd8a16aa1e7d92eb46 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:18:46.575283 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98923b4-37e0-4831-874f-2e4663c0a2dd None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:18:46.575613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98923b4-37e0-4831-874f-2e4663c0a2dd None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:18:46.575896 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98923b4-37e0-4831-874f-2e4663c0a2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:18:46.596039 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-b98923b4-37e0-4831-874f-2e4663c0a2dd None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:18:46.597329 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b98923b4-37e0-4831-874f-2e4663c0a2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:18:46.619212 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:18:46.687673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1126/2255] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:18:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:18:46.696197 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059fbf9-3337-4773-b800-6d22a6827985 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:18:46.696476 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059fbf9-3337-4773-b800-6d22a6827985 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:18:46.696730 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059fbf9-3337-4773-b800-6d22a6827985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:18:46.730342 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-3059fbf9-3337-4773-b800-6d22a6827985 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:18:46.732638 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-3059fbf9-3337-4773-b800-6d22a6827985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:18:46.744678 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1130/2256] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:18:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:18:46.749543 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98923b4-37e0-4831-874f-2e4663c0a2dd None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:18:46.762540 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98923b4-37e0-4831-874f-2e4663c0a2dd None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:18:46.768678 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830b26f6-fd11-4b27-9085-412df0f00fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:18:46.769649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b26f6-fd11-4b27-9085-412df0f00fa8 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:18:46.770038 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b26f6-fd11-4b27-9085-412df0f00fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:18:46.770385 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b26f6-fd11-4b27-9085-412df0f00fa8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:18:46.771281 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830b26f6-fd11-4b27-9085-412df0f00fa8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:18:46.772132 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830b26f6-fd11-4b27-9085-412df0f00fa8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:18:46.774071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830b26f6-fd11-4b27-9085-412df0f00fa8 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:18:46.788339 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1127/2257] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:18:46.794348 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059fbf9-3337-4773-b800-6d22a6827985 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:18:46.805131 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059fbf9-3337-4773-b800-6d22a6827985 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:18:46.810343 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90304f6-44d6-4b97-9e6b-d6c31da48692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:18:46.811090 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90304f6-44d6-4b97-9e6b-d6c31da48692 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:18:46.811374 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90304f6-44d6-4b97-9e6b-d6c31da48692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:18:46.811684 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90304f6-44d6-4b97-9e6b-d6c31da48692 admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:18:46.812410 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90304f6-44d6-4b97-9e6b-d6c31da48692 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:18:46.814406 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90304f6-44d6-4b97-9e6b-d6c31da48692 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:18:46.852063 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1131/2258] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:18:46 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:18:46.856829 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830b26f6-fd11-4b27-9085-412df0f00fa8 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:18:46.875154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830b26f6-fd11-4b27-9085-412df0f00fa8 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:18:46.977114 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9fccd2-7f4d-491b-ba1e-257a7735a242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:18:46.977894 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9fccd2-7f4d-491b-ba1e-257a7735a242 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:18:46.978190 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9fccd2-7f4d-491b-ba1e-257a7735a242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:18:46.978549 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9fccd2-7f4d-491b-ba1e-257a7735a242 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:18:46.979447 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9fccd2-7f4d-491b-ba1e-257a7735a242 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:18:46.981670 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9fccd2-7f4d-491b-ba1e-257a7735a242 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:18:47.020301 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1128/2259] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:18:46 2022] POST /identity/v3/users => generated 288 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:18:47.023608 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90304f6-44d6-4b97-9e6b-d6c31da48692 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:18:47.049160 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90304f6-44d6-4b97-9e6b-d6c31da48692 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:18:47.058364 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d612ccc-5bff-459f-a105-bb16e4c73e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:18:47.059966 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d612ccc-5bff-459f-a105-bb16e4c73e24 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:18:47.060727 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d612ccc-5bff-459f-a105-bb16e4c73e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:18:47.061525 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d612ccc-5bff-459f-a105-bb16e4c73e24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:18:47.063447 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d612ccc-5bff-459f-a105-bb16e4c73e24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:18:47.066956 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d612ccc-5bff-459f-a105-bb16e4c73e24 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:18:47.092965 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1132/2260] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:18:47.098158 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9fccd2-7f4d-491b-ba1e-257a7735a242 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:18:47.120671 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9fccd2-7f4d-491b-ba1e-257a7735a242 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:18:47.129647 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43894b8-4343-4266-97e2-a5905357c347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:18:47.131667 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43894b8-4343-4266-97e2-a5905357c347 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:18:47.132327 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43894b8-4343-4266-97e2-a5905357c347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:18:47.132940 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43894b8-4343-4266-97e2-a5905357c347 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:18:47.134484 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43894b8-4343-4266-97e2-a5905357c347 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:18:47.138186 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43894b8-4343-4266-97e2-a5905357c347 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:18:47.155592 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1129/2261] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:18:47.161319 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d612ccc-5bff-459f-a105-bb16e4c73e24 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:18:47.181233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d612ccc-5bff-459f-a105-bb16e4c73e24 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:18:47.187025 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f6c8d8-8e2f-45c4-98d6-8097227586fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:18:47.188080 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f6c8d8-8e2f-45c4-98d6-8097227586fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:18:47.188435 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f6c8d8-8e2f-45c4-98d6-8097227586fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:18:47.188938 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f6c8d8-8e2f-45c4-98d6-8097227586fd admin admin] PATH_INFO: `/v3/projects/92a927489f1f4358bceece0eb52deae0/users/2009c4582908459b8b9ecd22493fa838/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:18:47.208182 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f6c8d8-8e2f-45c4-98d6-8097227586fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=92a927489f1f4358bceece0eb52deae0, user_id=2009c4582908459b8b9ecd22493fa838, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:18:47.211855 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f6c8d8-8e2f-45c4-98d6-8097227586fd admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:18:47.227514 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1133/2262] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:18:47 2022] PUT /identity/v3/projects/92a927489f1f4358bceece0eb52deae0/users/2009c4582908459b8b9ecd22493fa838/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:18:47.238615 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd795f3-a346-4cf4-a0ef-61b428c559f7 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:18:47.238968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd795f3-a346-4cf4-a0ef-61b428c559f7 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:18:47.239219 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd795f3-a346-4cf4-a0ef-61b428c559f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:18:47.274481 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-cbd795f3-a346-4cf4-a0ef-61b428c559f7 None None] MFA Rules not processed for user `2009c4582908459b8b9ecd22493fa838`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:18:47.276899 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-cbd795f3-a346-4cf4-a0ef-61b428c559f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:18:47.360724 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1130/2263] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:18:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:18:47.373411 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f6c8d8-8e2f-45c4-98d6-8097227586fd admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:18:47.375222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-86f6c8d8-8e2f-45c4-98d6-8097227586fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:18:47.402626 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:18:47.445858 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f6c8d8-8e2f-45c4-98d6-8097227586fd admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:18:47.449927 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3706d3ac-f0a9-4c3d-a453-8b40d2a47563 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:18:47.450587 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3706d3ac-f0a9-4c3d-a453-8b40d2a47563 service cinder] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:18:47.450868 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3706d3ac-f0a9-4c3d-a453-8b40d2a47563 service cinder] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:18:47.451167 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3706d3ac-f0a9-4c3d-a453-8b40d2a47563 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:18:47.460616 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3706d3ac-f0a9-4c3d-a453-8b40d2a47563 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:18:47.462090 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3706d3ac-f0a9-4c3d-a453-8b40d2a47563 service cinder] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:18:47.472261 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1134/2264] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:18:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:18:48.736831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd795f3-a346-4cf4-a0ef-61b428c559f7 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:18:48.738611 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-cbd795f3-a346-4cf4-a0ef-61b428c559f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:18:48.763171 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:18:48.808831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd795f3-a346-4cf4-a0ef-61b428c559f7 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:18:48.813025 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20593e-eac9-460c-a000-a35193fe198b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:18:48.813875 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20593e-eac9-460c-a000-a35193fe198b service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:18:48.814220 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20593e-eac9-460c-a000-a35193fe198b service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:18:48.814584 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20593e-eac9-460c-a000-a35193fe198b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:18:48.816694 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6f20593e-eac9-460c-a000-a35193fe198b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:18:48.890362 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f20593e-eac9-460c-a000-a35193fe198b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:18:48.891878 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f20593e-eac9-460c-a000-a35193fe198b service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:18:48.908064 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1131/2265] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:18:48 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:18:57.467777 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=17 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:18:58.897824 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=42 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:19:00.535771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3706d3ac-f0a9-4c3d-a453-8b40d2a47563 service cinder] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:19:00.607501 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3706d3ac-f0a9-4c3d-a453-8b40d2a47563 service cinder] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:19:00.612360 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983e5dde-18cd-412e-a18c-c35fa437bb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:19:00.613257 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e5dde-18cd-412e-a18c-c35fa437bb7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:19:00.613541 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e5dde-18cd-412e-a18c-c35fa437bb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:19:00.613815 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e5dde-18cd-412e-a18c-c35fa437bb7e admin admin] PATH_INFO: `/v3/users/2009c4582908459b8b9ecd22493fa838` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:19:00.615597 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983e5dde-18cd-412e-a18c-c35fa437bb7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2009c4582908459b8b9ecd22493fa838)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:19:00.617629 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983e5dde-18cd-412e-a18c-c35fa437bb7e admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:19:00.682000 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-983e5dde-18cd-412e-a18c-c35fa437bb7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2009c4582908459b8b9ecd22493fa838'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:19:00.682657 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-983e5dde-18cd-412e-a18c-c35fa437bb7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2009c4582908459b8b9ecd22493fa838'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:19:00.694544 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-983e5dde-18cd-412e-a18c-c35fa437bb7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2009c4582908459b8b9ecd22493fa838'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:19:00.701595 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-983e5dde-18cd-412e-a18c-c35fa437bb7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2009c4582908459b8b9ecd22493fa838'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:19:00.702344 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-983e5dde-18cd-412e-a18c-c35fa437bb7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2009c4582908459b8b9ecd22493fa838'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:19:00.717104 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1135/2266] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:19:00 2022] DELETE /identity/v3/users/2009c4582908459b8b9ecd22493fa838 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:19:00.722152 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007d4d7-6ddf-463b-adc7-b59d5e840ba7 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:19:00.722685 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007d4d7-6ddf-463b-adc7-b59d5e840ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:19:00.723073 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007d4d7-6ddf-463b-adc7-b59d5e840ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:19:00.747270 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-d007d4d7-6ddf-463b-adc7-b59d5e840ba7 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:19:00.748561 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d007d4d7-6ddf-463b-adc7-b59d5e840ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:19:00.822946 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1132/2267] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:19:00.835318 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983e5dde-18cd-412e-a18c-c35fa437bb7e admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:19:00.836826 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-983e5dde-18cd-412e-a18c-c35fa437bb7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:19:00.902847 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983e5dde-18cd-412e-a18c-c35fa437bb7e admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:19:00.906622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c9789d-855d-41d4-8e6b-0b852cdb378c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:19:00.907281 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c9789d-855d-41d4-8e6b-0b852cdb378c service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:19:00.907599 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c9789d-855d-41d4-8e6b-0b852cdb378c service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:19:00.907875 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c9789d-855d-41d4-8e6b-0b852cdb378c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:19:00.918675 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c9789d-855d-41d4-8e6b-0b852cdb378c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:19:00.920189 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c9789d-855d-41d4-8e6b-0b852cdb378c service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:19:00.931204 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1136/2268] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:19:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:19:01.247655 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d007d4d7-6ddf-463b-adc7-b59d5e840ba7 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:19:01.250050 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d007d4d7-6ddf-463b-adc7-b59d5e840ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:19:01.268510 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d007d4d7-6ddf-463b-adc7-b59d5e840ba7 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:19:01.277459 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13499be5-c283-4f72-b723-f84806249cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:19:01.279036 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13499be5-c283-4f72-b723-f84806249cf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:19:01.279562 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13499be5-c283-4f72-b723-f84806249cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:19:01.280071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13499be5-c283-4f72-b723-f84806249cf8 admin admin] PATH_INFO: `/v3/projects/92a927489f1f4358bceece0eb52deae0` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:19:01.282438 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13499be5-c283-4f72-b723-f84806249cf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92a927489f1f4358bceece0eb52deae0)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:19:01.286052 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13499be5-c283-4f72-b723-f84806249cf8 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:19:01.345724 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-13499be5-c283-4f72-b723-f84806249cf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92a927489f1f4358bceece0eb52deae0', 'request_id': 'req-13499be5-c283-4f72-b723-f84806249cf8'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:19:01.353083 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-13499be5-c283-4f72-b723-f84806249cf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92a927489f1f4358bceece0eb52deae0', 'request_id': 'req-13499be5-c283-4f72-b723-f84806249cf8'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:19:01.365605 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-13499be5-c283-4f72-b723-f84806249cf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92a927489f1f4358bceece0eb52deae0', 'request_id': 'req-13499be5-c283-4f72-b723-f84806249cf8'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:19:01.379686 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-13499be5-c283-4f72-b723-f84806249cf8 admin admin] The token cache is being invalidate because project 92a927489f1f4358bceece0eb52deae0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:19:01.380488 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-13499be5-c283-4f72-b723-f84806249cf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:19:01.381191 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-13499be5-c283-4f72-b723-f84806249cf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:19:01.382797 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1133/2269] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:19:01 2022] DELETE /identity/v3/projects/92a927489f1f4358bceece0eb52deae0 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:19:01.388517 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e841121-c272-49fa-a24d-44cdafe81176 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:19:01.388804 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e841121-c272-49fa-a24d-44cdafe81176 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:19:01.389081 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e841121-c272-49fa-a24d-44cdafe81176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:19:01.408854 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-4e841121-c272-49fa-a24d-44cdafe81176 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:19:01.409991 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-4e841121-c272-49fa-a24d-44cdafe81176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:19:01.429064 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:19:01.481049 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1137/2270] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:19:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:19:01.548037 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a822013-2154-4f1f-82fd-3ae3631f8f2b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:19:01.548316 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a822013-2154-4f1f-82fd-3ae3631f8f2b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:19:01.548585 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a822013-2154-4f1f-82fd-3ae3631f8f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:19:01.569497 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-1a822013-2154-4f1f-82fd-3ae3631f8f2b None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:19:01.570750 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1a822013-2154-4f1f-82fd-3ae3631f8f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:19:01.579385 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1134/2271] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:19:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:19:01.584123 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e841121-c272-49fa-a24d-44cdafe81176 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:19:01.598583 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e841121-c272-49fa-a24d-44cdafe81176 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:19:01.604379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e59d6e-657f-4179-aabb-28d8edcac121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:19:01.605088 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e59d6e-657f-4179-aabb-28d8edcac121 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:19:01.605370 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e59d6e-657f-4179-aabb-28d8edcac121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:19:01.605642 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e59d6e-657f-4179-aabb-28d8edcac121 admin admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:19:01.606388 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e59d6e-657f-4179-aabb-28d8edcac121 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:19:01.606690 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e59d6e-657f-4179-aabb-28d8edcac121 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:19:01.608415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e59d6e-657f-4179-aabb-28d8edcac121 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:19:01.621511 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1138/2272] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:19:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:19:01.625904 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a822013-2154-4f1f-82fd-3ae3631f8f2b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:19:01.637661 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a822013-2154-4f1f-82fd-3ae3631f8f2b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:19:01.643107 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5ff153-62ff-43bd-af83-a97783e01871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:19:01.643927 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ff153-62ff-43bd-af83-a97783e01871 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:19:01.644258 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ff153-62ff-43bd-af83-a97783e01871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:19:01.644581 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ff153-62ff-43bd-af83-a97783e01871 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:19:01.645420 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5ff153-62ff-43bd-af83-a97783e01871 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:19:01.647556 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5ff153-62ff-43bd-af83-a97783e01871 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:19:01.682135 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1135/2273] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:19:01 2022] POST /identity/v3/projects => generated 379 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:19:01.686710 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e59d6e-657f-4179-aabb-28d8edcac121 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:19:01.699000 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e59d6e-657f-4179-aabb-28d8edcac121 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:19:01.776373 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04747774-00a3-456f-b2de-1403cf1b76f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:19:01.777226 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04747774-00a3-456f-b2de-1403cf1b76f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:19:01.777511 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04747774-00a3-456f-b2de-1403cf1b76f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:19:01.777817 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04747774-00a3-456f-b2de-1403cf1b76f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:19:01.778540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04747774-00a3-456f-b2de-1403cf1b76f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:19:01.780765 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04747774-00a3-456f-b2de-1403cf1b76f4 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:19:01.805181 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1139/2274] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:19:01 2022] POST /identity/v3/users => generated 290 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:19:01.809765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5ff153-62ff-43bd-af83-a97783e01871 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:19:01.821687 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5ff153-62ff-43bd-af83-a97783e01871 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:19:01.827136 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e122004b-6559-4daa-ab53-fdf540dee45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:19:01.828037 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122004b-6559-4daa-ab53-fdf540dee45f admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:19:01.828327 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122004b-6559-4daa-ab53-fdf540dee45f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:19:01.828685 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122004b-6559-4daa-ab53-fdf540dee45f admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:19:01.829601 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e122004b-6559-4daa-ab53-fdf540dee45f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:19:01.831447 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e122004b-6559-4daa-ab53-fdf540dee45f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:19:01.844426 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1136/2275] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:19:01.851671 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04747774-00a3-456f-b2de-1403cf1b76f4 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:19:01.870010 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04747774-00a3-456f-b2de-1403cf1b76f4 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:19:01.878602 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9ead4-2c00-4e23-a0df-4c4402866ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:19:01.879716 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9ead4-2c00-4e23-a0df-4c4402866ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:19:01.880210 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9ead4-2c00-4e23-a0df-4c4402866ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:19:01.880718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9ead4-2c00-4e23-a0df-4c4402866ea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:19:01.882030 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d9ead4-2c00-4e23-a0df-4c4402866ea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:19:01.885289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d9ead4-2c00-4e23-a0df-4c4402866ea7 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:19:01.902219 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1140/2276] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:19:01.907564 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e122004b-6559-4daa-ab53-fdf540dee45f admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:19:01.919013 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e122004b-6559-4daa-ab53-fdf540dee45f admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:19:01.923946 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faac9c1-4559-4bc6-a5ac-c5e1fb6e4055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:19:01.924860 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faac9c1-4559-4bc6-a5ac-c5e1fb6e4055 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:19:01.925399 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faac9c1-4559-4bc6-a5ac-c5e1fb6e4055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:19:01.925677 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faac9c1-4559-4bc6-a5ac-c5e1fb6e4055 admin admin] PATH_INFO: `/v3/projects/63fee05e13f74a46ab5f44c0a393220b/users/4fb52c7a4e8047c4aa8d2d86f3c30a35/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:19:01.943499 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faac9c1-4559-4bc6-a5ac-c5e1fb6e4055 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63fee05e13f74a46ab5f44c0a393220b, user_id=4fb52c7a4e8047c4aa8d2d86f3c30a35, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:19:01.946171 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faac9c1-4559-4bc6-a5ac-c5e1fb6e4055 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:19:01.963564 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1137/2277] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:19:01 2022] PUT /identity/v3/projects/63fee05e13f74a46ab5f44c0a393220b/users/4fb52c7a4e8047c4aa8d2d86f3c30a35/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:19:01.969002 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cbf8e3-bbe1-40bd-98dc-b4d59f6ad744 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:19:01.969410 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cbf8e3-bbe1-40bd-98dc-b4d59f6ad744 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:19:01.969758 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cbf8e3-bbe1-40bd-98dc-b4d59f6ad744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:19:01.991121 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-22cbf8e3-bbe1-40bd-98dc-b4d59f6ad744 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:19:01.992430 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-22cbf8e3-bbe1-40bd-98dc-b4d59f6ad744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:19:02.057763 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1141/2278] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:19:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:19:02.067782 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faac9c1-4559-4bc6-a5ac-c5e1fb6e4055 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:19:02.069597 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9faac9c1-4559-4bc6-a5ac-c5e1fb6e4055 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:19:02.092761 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:19:02.134479 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faac9c1-4559-4bc6-a5ac-c5e1fb6e4055 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:19:02.138305 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabd5aa-ca49-4920-a223-73fe7304969f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:19:02.138969 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabd5aa-ca49-4920-a223-73fe7304969f service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:19:02.139250 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabd5aa-ca49-4920-a223-73fe7304969f service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:19:02.139525 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabd5aa-ca49-4920-a223-73fe7304969f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:19:02.150001 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fabd5aa-ca49-4920-a223-73fe7304969f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:19:02.151463 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fabd5aa-ca49-4920-a223-73fe7304969f service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:19:02.163051 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1138/2279] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:19:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:19:02.859964 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f327ec6-004d-40bf-98e8-c2b0456db0c6 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:19:02.860432 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f327ec6-004d-40bf-98e8-c2b0456db0c6 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:19:02.860619 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f327ec6-004d-40bf-98e8-c2b0456db0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:19:02.889520 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-0f327ec6-004d-40bf-98e8-c2b0456db0c6 None None] MFA Rules not processed for user `4fb52c7a4e8047c4aa8d2d86f3c30a35`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:19:02.890852 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-0f327ec6-004d-40bf-98e8-c2b0456db0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:19:02.910900 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:19:02.994697 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1142/2280] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:19:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:19:03.004378 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabd5aa-ca49-4920-a223-73fe7304969f service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:19:03.007404 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-2fabd5aa-ca49-4920-a223-73fe7304969f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:19:03.105164 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabd5aa-ca49-4920-a223-73fe7304969f service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:19:03.108812 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744e17f-5418-4a33-a75d-4d1c4a5b5301 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:19:03.109395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744e17f-5418-4a33-a75d-4d1c4a5b5301 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:19:03.109671 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744e17f-5418-4a33-a75d-4d1c4a5b5301 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:19:03.109938 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744e17f-5418-4a33-a75d-4d1c4a5b5301 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:19:03.119369 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6744e17f-5418-4a33-a75d-4d1c4a5b5301 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:19:03.120855 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6744e17f-5418-4a33-a75d-4d1c4a5b5301 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:19:03.130756 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1139/2281] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:19:03 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:19:04.524619 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bffe941-a8ff-42c5-b353-e453158a8359 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:19:04.525171 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bffe941-a8ff-42c5-b353-e453158a8359 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:19:04.525418 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bffe941-a8ff-42c5-b353-e453158a8359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:19:04.544140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-6bffe941-a8ff-42c5-b353-e453158a8359 None None] MFA Rules not processed for user `4fb52c7a4e8047c4aa8d2d86f3c30a35`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:19:04.545361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-6bffe941-a8ff-42c5-b353-e453158a8359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:19:04.554478 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1143/2282] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:19:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:19:13.125932 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=43 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:19:14.538374 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=22 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:19:31.650333 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce17891-8270-4bec-ae46-a90248816d04 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:19:31.650915 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce17891-8270-4bec-ae46-a90248816d04 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:19:31.650915 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce17891-8270-4bec-ae46-a90248816d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:19:31.669270 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-8ce17891-8270-4bec-ae46-a90248816d04 None None] MFA Rules not processed for user `5e052654361f4580ab36a347610f63c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:19:31.670431 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8ce17891-8270-4bec-ae46-a90248816d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:19:31.739390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1140/2283] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:19:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:19:41.730476 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=25 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:19:50.651448 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2a8c3-1386-4e6b-8e82-136f2c28a7b6 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:19:50.652057 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2a8c3-1386-4e6b-8e82-136f2c28a7b6 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:19:50.652284 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2a8c3-1386-4e6b-8e82-136f2c28a7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:19:50.683033 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-2ec2a8c3-1386-4e6b-8e82-136f2c28a7b6 None None] MFA Rules not processed for user `a32702a67a7b49ffa153fc59251fd169`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:19:50.685573 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-2ec2a8c3-1386-4e6b-8e82-136f2c28a7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:19:50.795878 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1144/2284] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Jul 13 23:19:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:19:50.862549 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce17891-8270-4bec-ae46-a90248816d04 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:19:50.864053 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8ce17891-8270-4bec-ae46-a90248816d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:19:50.984994 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce17891-8270-4bec-ae46-a90248816d04 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:19:50.991285 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d65921-c0d3-475e-9177-40b118e415d3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786fc44611804d3ca6280ffcab415569', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:19:50.992206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d65921-c0d3-475e-9177-40b118e415d3 service swift] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:19:50.992491 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d65921-c0d3-475e-9177-40b118e415d3 service swift] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:19:50.992768 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d65921-c0d3-475e-9177-40b118e415d3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:19:51.003007 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d65921-c0d3-475e-9177-40b118e415d3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:19:51.004715 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d65921-c0d3-475e-9177-40b118e415d3 service swift] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:19:51.015386 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1141/2285] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Jul 13 23:19:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 23:20:00.772316 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=25 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:20:01.011335 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=32 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:20:18.660838 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5890d-a804-4e7e-9880-a4ea9161e40d None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:20:18.662056 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5890d-a804-4e7e-9880-a4ea9161e40d None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:20:18.662056 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5890d-a804-4e7e-9880-a4ea9161e40d None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:20:18.663668 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1145/2286] 10.222.0.60 () {54 vars in 921 bytes} [Wed Jul 13 23:20:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:20:18.671307 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c89d5-8d40-4f0b-b8cf-d7414041a5d6 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:20:18.671962 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c89d5-8d40-4f0b-b8cf-d7414041a5d6 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:20:18.671962 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c89d5-8d40-4f0b-b8cf-d7414041a5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:20:18.700769 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-008c89d5-8d40-4f0b-b8cf-d7414041a5d6 None None] MFA Rules not processed for user `4789384d158b4995bd30e1e37e30f5f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:20:18.702742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-008c89d5-8d40-4f0b-b8cf-d7414041a5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:20:18.807219 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1142/2287] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Jul 13 23:20:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:20:18.819580 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c5890d-a804-4e7e-9880-a4ea9161e40d None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:20:18.821991 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-f5c5890d-a804-4e7e-9880-a4ea9161e40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:20:18.897187 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c5890d-a804-4e7e-9880-a4ea9161e40d None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:20:18.901233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad0cdf8-7f3f-4e64-ad00-c9e34ed534f2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ce9e6497fdb4068b23aa3f7b2c19536', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:20:18.901919 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad0cdf8-7f3f-4e64-ad00-c9e34ed534f2 service glance] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:20:18.902205 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad0cdf8-7f3f-4e64-ad00-c9e34ed534f2 service glance] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:20:18.902482 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad0cdf8-7f3f-4e64-ad00-c9e34ed534f2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:20:18.915047 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad0cdf8-7f3f-4e64-ad00-c9e34ed534f2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:20:18.916644 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad0cdf8-7f3f-4e64-ad00-c9e34ed534f2 service glance] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:20:18.930875 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1146/2288] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:20:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:20:19.003705 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7239ce-129c-46ad-bdaa-8c1429919f9d None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:20:19.004003 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7239ce-129c-46ad-bdaa-8c1429919f9d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:20:19.004280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7239ce-129c-46ad-bdaa-8c1429919f9d None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:20:19.006270 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1143/2289] 10.222.0.60 () {54 vars in 921 bytes} [Wed Jul 13 23:20:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:20:19.012342 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18e9f4-8171-4c25-afb2-f5fce107fe6c None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:20:19.012793 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18e9f4-8171-4c25-afb2-f5fce107fe6c None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:20:19.013227 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18e9f4-8171-4c25-afb2-f5fce107fe6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:20:19.032878 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-1f18e9f4-8171-4c25-afb2-f5fce107fe6c None None] MFA Rules not processed for user `2d670df2aff5478ebf61f55358d554de`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:20:19.034141 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-1f18e9f4-8171-4c25-afb2-f5fce107fe6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:20:19.041801 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1147/2290] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Jul 13 23:20:19 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:20:19.055293 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7239ce-129c-46ad-bdaa-8c1429919f9d None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:20:19.074511 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7239ce-129c-46ad-bdaa-8c1429919f9d None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:20:19.080690 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5435e26-25ff-4226-a19b-dcc00ca710f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:20:19.081840 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5435e26-25ff-4226-a19b-dcc00ca710f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:20:19.082368 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5435e26-25ff-4226-a19b-dcc00ca710f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:20:19.082883 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5435e26-25ff-4226-a19b-dcc00ca710f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:20:19.098395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5435e26-25ff-4226-a19b-dcc00ca710f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:20:19.101456 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5435e26-25ff-4226-a19b-dcc00ca710f5 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:20:19.114159 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1144/2291] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:20:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:20:19.221467 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4649bc-331f-4bb7-bd09-1671fa484388 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:20:19.221909 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4649bc-331f-4bb7-bd09-1671fa484388 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:20:19.222328 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4649bc-331f-4bb7-bd09-1671fa484388 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:20:19.224319 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1148/2292] 10.222.0.60 () {54 vars in 921 bytes} [Wed Jul 13 23:20:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:20:19.231659 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b217b-cfdf-46d6-9a4e-754d82e7a4d1 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:20:19.232071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b217b-cfdf-46d6-9a4e-754d82e7a4d1 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:20:19.232504 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b217b-cfdf-46d6-9a4e-754d82e7a4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:20:19.253197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-e29b217b-cfdf-46d6-9a4e-754d82e7a4d1 None None] MFA Rules not processed for user `4789384d158b4995bd30e1e37e30f5f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:20:19.254445 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e29b217b-cfdf-46d6-9a4e-754d82e7a4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:20:19.264933 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1145/2293] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Jul 13 23:20:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:20:19.277051 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4649bc-331f-4bb7-bd09-1671fa484388 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:20:19.289945 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4649bc-331f-4bb7-bd09-1671fa484388 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:20:19.293823 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c96db-671d-4c3b-b32e-1b63bf4b5829 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:20:19.294605 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c96db-671d-4c3b-b32e-1b63bf4b5829 service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:20:19.294919 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c96db-671d-4c3b-b32e-1b63bf4b5829 service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:20:19.295235 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c96db-671d-4c3b-b32e-1b63bf4b5829 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:20:19.308195 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97c96db-671d-4c3b-b32e-1b63bf4b5829 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:20:19.309610 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97c96db-671d-4c3b-b32e-1b63bf4b5829 service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:20:19.323862 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1149/2294] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:20:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:20:24.691327 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47579c5c-27cc-45cc-9aed-a7afd86b2ca7 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:20:24.694287 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47579c5c-27cc-45cc-9aed-a7afd86b2ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:20:24.694287 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47579c5c-27cc-45cc-9aed-a7afd86b2ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:20:24.718489 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-47579c5c-27cc-45cc-9aed-a7afd86b2ca7 None None] MFA Rules not processed for user `a32702a67a7b49ffa153fc59251fd169`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:20:24.719763 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-47579c5c-27cc-45cc-9aed-a7afd86b2ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:20:24.727836 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1146/2295] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Jul 13 23:20:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:20:24.744870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c96db-671d-4c3b-b32e-1b63bf4b5829 service neutron] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:20:24.761561 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c96db-671d-4c3b-b32e-1b63bf4b5829 service neutron] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:20:24.769250 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956cbcb8-ad0d-4917-ba6c-434b885d45ab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786fc44611804d3ca6280ffcab415569', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:20:24.770231 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956cbcb8-ad0d-4917-ba6c-434b885d45ab service swift] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:20:24.771171 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956cbcb8-ad0d-4917-ba6c-434b885d45ab service swift] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:20:24.771605 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956cbcb8-ad0d-4917-ba6c-434b885d45ab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:20:24.781394 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956cbcb8-ad0d-4917-ba6c-434b885d45ab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:20:24.783001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956cbcb8-ad0d-4917-ba6c-434b885d45ab service swift] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:20:24.792957 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1150/2296] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Jul 13 23:20:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 13 23:20:27.288094 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47579c5c-27cc-45cc-9aed-a7afd86b2ca7 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:20:27.289698 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-47579c5c-27cc-45cc-9aed-a7afd86b2ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:20:27.307293 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47579c5c-27cc-45cc-9aed-a7afd86b2ca7 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:20:27.315705 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c4ebd9-1da9-413a-836f-93c770c6b4b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:20:27.316379 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4ebd9-1da9-413a-836f-93c770c6b4b6 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:20:27.316659 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4ebd9-1da9-413a-836f-93c770c6b4b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:20:27.316940 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4ebd9-1da9-413a-836f-93c770c6b4b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:20:27.318923 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-26c4ebd9-1da9-413a-836f-93c770c6b4b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:20:27.407133 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c4ebd9-1da9-413a-836f-93c770c6b4b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:20:27.410464 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c4ebd9-1da9-413a-836f-93c770c6b4b6 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:20:27.429341 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1147/2297] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:20:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:20:30.455815 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956cbcb8-ad0d-4917-ba6c-434b885d45ab service swift] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:20:30.457653 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-956cbcb8-ad0d-4917-ba6c-434b885d45ab service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:20:30.470439 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956cbcb8-ad0d-4917-ba6c-434b885d45ab service swift] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:20:30.474890 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e1c80-ff8a-4e3f-a30a-fb9f569cdee5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:20:30.475650 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e1c80-ff8a-4e3f-a30a-fb9f569cdee5 service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:20:30.475979 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e1c80-ff8a-4e3f-a30a-fb9f569cdee5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:20:30.476302 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e1c80-ff8a-4e3f-a30a-fb9f569cdee5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:20:30.478335 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-e39e1c80-ff8a-4e3f-a30a-fb9f569cdee5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:20:30.495026 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39e1c80-ff8a-4e3f-a30a-fb9f569cdee5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:20:30.496783 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39e1c80-ff8a-4e3f-a30a-fb9f569cdee5 service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:20:30.509742 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1151/2298] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:20:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:20:33.588188 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c4ebd9-1da9-413a-836f-93c770c6b4b6 service nova] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:20:33.599319 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c4ebd9-1da9-413a-836f-93c770c6b4b6 service nova] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:20:33.603057 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696fda8-a9b0-4156-93f3-4c5a6e158020 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:20:33.603743 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696fda8-a9b0-4156-93f3-4c5a6e158020 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:20:33.604080 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696fda8-a9b0-4156-93f3-4c5a6e158020 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:20:33.604355 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696fda8-a9b0-4156-93f3-4c5a6e158020 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:20:33.606243 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8696fda8-a9b0-4156-93f3-4c5a6e158020 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:20:33.637828 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8696fda8-a9b0-4156-93f3-4c5a6e158020 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:20:33.639301 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8696fda8-a9b0-4156-93f3-4c5a6e158020 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:20:33.656516 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1148/2299] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:20:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:20:40.503829 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=47 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:20:43.646951 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=77 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:21:01.432288 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92c951-07d6-4b2b-8dc6-78068bffb6de None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:21:01.432847 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92c951-07d6-4b2b-8dc6-78068bffb6de None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:21:01.432948 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92c951-07d6-4b2b-8dc6-78068bffb6de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:21:01.452868 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-4e92c951-07d6-4b2b-8dc6-78068bffb6de None None] MFA Rules not processed for user `4fb52c7a4e8047c4aa8d2d86f3c30a35`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:21:01.454098 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-4e92c951-07d6-4b2b-8dc6-78068bffb6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:21:01.461238 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1152/2300] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:21:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:21:01.693542 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696fda8-a9b0-4156-93f3-4c5a6e158020 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:21:01.702095 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696fda8-a9b0-4156-93f3-4c5a6e158020 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:21:01.705709 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c784e2-276e-457f-b240-bbd65e40effb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:21:01.706366 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c784e2-276e-457f-b240-bbd65e40effb service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:21:01.706656 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c784e2-276e-457f-b240-bbd65e40effb service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:21:01.706934 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c784e2-276e-457f-b240-bbd65e40effb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:21:01.708726 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-87c784e2-276e-457f-b240-bbd65e40effb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:21:01.726900 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c784e2-276e-457f-b240-bbd65e40effb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:21:01.728373 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c784e2-276e-457f-b240-bbd65e40effb service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:21:01.736427 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1149/2301] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:21:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:21:02.057887 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e92c951-07d6-4b2b-8dc6-78068bffb6de None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:21:02.057887 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-4e92c951-07d6-4b2b-8dc6-78068bffb6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:21:02.132179 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e92c951-07d6-4b2b-8dc6-78068bffb6de None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:21:02.137718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f612444-c3be-407c-9fad-10a0fe01345a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64824ba84a024930bbc177f8a8d9a046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:21:02.138375 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f612444-c3be-407c-9fad-10a0fe01345a service placement] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:21:02.138653 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f612444-c3be-407c-9fad-10a0fe01345a service placement] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:21:02.138924 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f612444-c3be-407c-9fad-10a0fe01345a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:21:02.140787 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-0f612444-c3be-407c-9fad-10a0fe01345a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:21:02.153434 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f612444-c3be-407c-9fad-10a0fe01345a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:21:02.155062 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f612444-c3be-407c-9fad-10a0fe01345a service placement] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:21:02.166167 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1153/2302] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:21:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:21:03.898697 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c784e2-276e-457f-b240-bbd65e40effb service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:21:03.908220 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c784e2-276e-457f-b240-bbd65e40effb service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:21:03.912746 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a332a-6414-4707-9a46-f0fa5ab01f80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:21:03.913385 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a332a-6414-4707-9a46-f0fa5ab01f80 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:21:03.913672 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a332a-6414-4707-9a46-f0fa5ab01f80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:21:03.913945 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a332a-6414-4707-9a46-f0fa5ab01f80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:21:03.915755 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1d2a332a-6414-4707-9a46-f0fa5ab01f80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:21:03.917307 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f612444-c3be-407c-9fad-10a0fe01345a service placement] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:21:03.926041 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2a332a-6414-4707-9a46-f0fa5ab01f80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:21:03.927442 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f612444-c3be-407c-9fad-10a0fe01345a service placement] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:21:03.927606 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2a332a-6414-4707-9a46-f0fa5ab01f80 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:21:03.931466 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4375770d-6a39-415f-9588-bb07c78c34be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:21:03.932220 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375770d-6a39-415f-9588-bb07c78c34be service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:21:03.932530 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375770d-6a39-415f-9588-bb07c78c34be service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:21:03.932806 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375770d-6a39-415f-9588-bb07c78c34be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:21:03.936060 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1150/2303] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:21:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:21:03.941121 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4375770d-6a39-415f-9588-bb07c78c34be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:21:03.942579 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4375770d-6a39-415f-9588-bb07c78c34be service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:21:03.950959 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1154/2304] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:21:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:21:04.652914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a332a-6414-4707-9a46-f0fa5ab01f80 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:21:04.654863 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1d2a332a-6414-4707-9a46-f0fa5ab01f80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:21:04.675205 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a332a-6414-4707-9a46-f0fa5ab01f80 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:21:04.680992 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac5ad4-8811-487d-96e6-84078a4b9772 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:21:04.681905 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac5ad4-8811-487d-96e6-84078a4b9772 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:21:04.682179 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac5ad4-8811-487d-96e6-84078a4b9772 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:21:04.682572 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac5ad4-8811-487d-96e6-84078a4b9772 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:21:04.684439 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4cac5ad4-8811-487d-96e6-84078a4b9772 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:21:04.698180 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cac5ad4-8811-487d-96e6-84078a4b9772 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:21:04.699968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cac5ad4-8811-487d-96e6-84078a4b9772 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:21:04.711684 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1151/2305] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:21:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:21:10.247522 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e7c7e8-9acb-40c5-ba61-c71ccb965c2c None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:21:10.248499 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e7c7e8-9acb-40c5-ba61-c71ccb965c2c None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:21:10.248804 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e7c7e8-9acb-40c5-ba61-c71ccb965c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:21:10.285929 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-a8e7c7e8-9acb-40c5-ba61-c71ccb965c2c None None] MFA Rules not processed for user `a32702a67a7b49ffa153fc59251fd169`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:21:10.287409 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-a8e7c7e8-9acb-40c5-ba61-c71ccb965c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:21:10.301940 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1155/2306] 10.222.0.60 () {58 vars in 967 bytes} [Wed Jul 13 23:21:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:21:10.318611 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac5ad4-8811-487d-96e6-84078a4b9772 service nova] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:21:10.330105 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac5ad4-8811-487d-96e6-84078a4b9772 service nova] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:21:10.336035 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8460a2-1cd3-4501-8115-b95344093529 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786fc44611804d3ca6280ffcab415569', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:21:10.338741 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8460a2-1cd3-4501-8115-b95344093529 service swift] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:21:10.339030 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8460a2-1cd3-4501-8115-b95344093529 service swift] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:21:10.339307 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8460a2-1cd3-4501-8115-b95344093529 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:21:10.348691 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8460a2-1cd3-4501-8115-b95344093529 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:21:10.350288 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8460a2-1cd3-4501-8115-b95344093529 service swift] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:21:10.358620 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1152/2307] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Jul 13 23:21:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 23:21:20.279487 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=44 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:21:20.355307 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=24 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:21:59.877707 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e7c7e8-9acb-40c5-ba61-c71ccb965c2c None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:21:59.890969 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e7c7e8-9acb-40c5-ba61-c71ccb965c2c None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:21:59.896409 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e4aa37-b415-45a7-a80b-ddf76d27f3ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:21:59.897048 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4aa37-b415-45a7-a80b-ddf76d27f3ff service nova] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:21:59.897330 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4aa37-b415-45a7-a80b-ddf76d27f3ff service nova] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:21:59.897603 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4aa37-b415-45a7-a80b-ddf76d27f3ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:21:59.899371 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-79e4aa37-b415-45a7-a80b-ddf76d27f3ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:21:59.957408 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e4aa37-b415-45a7-a80b-ddf76d27f3ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:21:59.958976 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e4aa37-b415-45a7-a80b-ddf76d27f3ff service nova] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:21:59.969444 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1156/2308] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:21:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:22:09.966011 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=28 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:22:11.665636 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a902d0f1-f5e0-4fce-befc-f99b4e585648 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:22:11.666164 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a902d0f1-f5e0-4fce-befc-f99b4e585648 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:22:11.666360 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a902d0f1-f5e0-4fce-befc-f99b4e585648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:22:11.686074 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-a902d0f1-f5e0-4fce-befc-f99b4e585648 None None] MFA Rules not processed for user `c3871ceed47e4fcea07c5f879c8462cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:22:11.687407 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a902d0f1-f5e0-4fce-befc-f99b4e585648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:22:11.748225 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1153/2309] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:22:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:22:12.066039 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e4aa37-b415-45a7-a80b-ddf76d27f3ff service nova] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:22:12.075546 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e4aa37-b415-45a7-a80b-ddf76d27f3ff service nova] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:22:12.079151 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cf33e-9603-4f7d-8467-a35436aa1933 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:22:12.079829 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cf33e-9603-4f7d-8467-a35436aa1933 service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:22:12.080112 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cf33e-9603-4f7d-8467-a35436aa1933 service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:22:12.080390 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cf33e-9603-4f7d-8467-a35436aa1933 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:22:12.082158 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-7f3cf33e-9603-4f7d-8467-a35436aa1933 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:22:12.091756 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3cf33e-9603-4f7d-8467-a35436aa1933 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:22:12.093393 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3cf33e-9603-4f7d-8467-a35436aa1933 service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:22:12.102387 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1157/2310] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:22:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:22:14.311960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a902d0f1-f5e0-4fce-befc-f99b4e585648 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:22:14.313531 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a902d0f1-f5e0-4fce-befc-f99b4e585648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:22:14.329022 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a902d0f1-f5e0-4fce-befc-f99b4e585648 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:22:14.334210 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da523648-b1ba-41a8-b041-423d8db828da service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64824ba84a024930bbc177f8a8d9a046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:22:14.334935 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da523648-b1ba-41a8-b041-423d8db828da service placement] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:22:14.335302 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da523648-b1ba-41a8-b041-423d8db828da service placement] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:22:14.335693 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da523648-b1ba-41a8-b041-423d8db828da service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:22:14.337553 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-da523648-b1ba-41a8-b041-423d8db828da service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:22:14.349938 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da523648-b1ba-41a8-b041-423d8db828da service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:22:14.351629 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da523648-b1ba-41a8-b041-423d8db828da service placement] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:22:14.370555 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1154/2311] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:22:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:22:14.413355 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cf33e-9603-4f7d-8467-a35436aa1933 service neutron] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:22:14.428240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cf33e-9603-4f7d-8467-a35436aa1933 service neutron] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:22:14.435005 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06846074-f786-4215-998b-cedb6a389dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:22:14.436656 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06846074-f786-4215-998b-cedb6a389dcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:22:14.436949 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06846074-f786-4215-998b-cedb6a389dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:22:14.437227 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06846074-f786-4215-998b-cedb6a389dcc admin admin] PATH_INFO: `/v3/users/4fb52c7a4e8047c4aa8d2d86f3c30a35` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:22:14.438513 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06846074-f786-4215-998b-cedb6a389dcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fb52c7a4e8047c4aa8d2d86f3c30a35)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:22:14.440905 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06846074-f786-4215-998b-cedb6a389dcc admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:22:14.501350 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-06846074-f786-4215-998b-cedb6a389dcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fb52c7a4e8047c4aa8d2d86f3c30a35'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:22:14.502010 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-06846074-f786-4215-998b-cedb6a389dcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fb52c7a4e8047c4aa8d2d86f3c30a35'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:22:14.515961 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-06846074-f786-4215-998b-cedb6a389dcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fb52c7a4e8047c4aa8d2d86f3c30a35'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:22:14.523897 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-06846074-f786-4215-998b-cedb6a389dcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fb52c7a4e8047c4aa8d2d86f3c30a35'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:22:14.524685 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-06846074-f786-4215-998b-cedb6a389dcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fb52c7a4e8047c4aa8d2d86f3c30a35'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:22:14.551276 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1158/2312] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:22:14 2022] DELETE /identity/v3/users/4fb52c7a4e8047c4aa8d2d86f3c30a35 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:22:14.753233 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da523648-b1ba-41a8-b041-423d8db828da service placement] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:22:14.767650 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da523648-b1ba-41a8-b041-423d8db828da service placement] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:22:14.853647 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4cafb2-b8c6-4b25-970a-9b813b3af1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:22:14.855133 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4cafb2-b8c6-4b25-970a-9b813b3af1d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:22:14.855424 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4cafb2-b8c6-4b25-970a-9b813b3af1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:22:14.855918 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4cafb2-b8c6-4b25-970a-9b813b3af1d0 admin admin] PATH_INFO: `/v3/projects/63fee05e13f74a46ab5f44c0a393220b` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:22:14.857957 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4cafb2-b8c6-4b25-970a-9b813b3af1d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63fee05e13f74a46ab5f44c0a393220b)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:22:14.865407 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4cafb2-b8c6-4b25-970a-9b813b3af1d0 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:22:14.934019 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-fe4cafb2-b8c6-4b25-970a-9b813b3af1d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63fee05e13f74a46ab5f44c0a393220b', 'request_id': 'req-fe4cafb2-b8c6-4b25-970a-9b813b3af1d0'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:22:14.941581 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-fe4cafb2-b8c6-4b25-970a-9b813b3af1d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63fee05e13f74a46ab5f44c0a393220b', 'request_id': 'req-fe4cafb2-b8c6-4b25-970a-9b813b3af1d0'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:22:14.952672 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-fe4cafb2-b8c6-4b25-970a-9b813b3af1d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63fee05e13f74a46ab5f44c0a393220b', 'request_id': 'req-fe4cafb2-b8c6-4b25-970a-9b813b3af1d0'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:22:14.966964 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-fe4cafb2-b8c6-4b25-970a-9b813b3af1d0 admin admin] The token cache is being invalidate because project 63fee05e13f74a46ab5f44c0a393220b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:22:14.967233 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-fe4cafb2-b8c6-4b25-970a-9b813b3af1d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:22:14.972733 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-fe4cafb2-b8c6-4b25-970a-9b813b3af1d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:22:14.974787 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1155/2313] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:22:14 2022] DELETE /identity/v3/projects/63fee05e13f74a46ab5f44c0a393220b => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:22:24.541846 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=35,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:22:24.963208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=82,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:22:27.783657 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06846074-f786-4215-998b-cedb6a389dcc admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:22:27.785308 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-06846074-f786-4215-998b-cedb6a389dcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:22:27.875791 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06846074-f786-4215-998b-cedb6a389dcc admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:22:27.947519 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6803920-ddde-496a-b61b-78909d1b7fc8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:22:27.948218 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6803920-ddde-496a-b61b-78909d1b7fc8 service nova] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:22:27.948505 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6803920-ddde-496a-b61b-78909d1b7fc8 service nova] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:22:27.948782 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6803920-ddde-496a-b61b-78909d1b7fc8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:22:27.950633 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-f6803920-ddde-496a-b61b-78909d1b7fc8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:22:27.963885 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6803920-ddde-496a-b61b-78909d1b7fc8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:22:27.965605 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6803920-ddde-496a-b61b-78909d1b7fc8 service nova] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:22:27.977213 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1159/2314] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:22:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:22:28.949040 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4cafb2-b8c6-4b25-970a-9b813b3af1d0 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:22:28.950589 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-fe4cafb2-b8c6-4b25-970a-9b813b3af1d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:22:28.967052 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4cafb2-b8c6-4b25-970a-9b813b3af1d0 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:22:28.972507 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce2325-6eda-4daf-bdfb-9a307d045f7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:22:28.973133 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce2325-6eda-4daf-bdfb-9a307d045f7b service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:22:28.973417 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce2325-6eda-4daf-bdfb-9a307d045f7b service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:22:28.973695 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce2325-6eda-4daf-bdfb-9a307d045f7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:22:28.976028 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-61ce2325-6eda-4daf-bdfb-9a307d045f7b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:22:29.033605 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce2325-6eda-4daf-bdfb-9a307d045f7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:22:29.035197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce2325-6eda-4daf-bdfb-9a307d045f7b service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:22:29.044496 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1156/2315] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:22:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:22:29.120776 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6803920-ddde-496a-b61b-78909d1b7fc8 service nova] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:22:29.122300 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-f6803920-ddde-496a-b61b-78909d1b7fc8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:22:29.136580 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6803920-ddde-496a-b61b-78909d1b7fc8 service nova] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:22:29.142024 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbe875-e91f-4894-89ff-3834f37b91f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:22:29.142665 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbe875-e91f-4894-89ff-3834f37b91f7 service nova] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:22:29.142944 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbe875-e91f-4894-89ff-3834f37b91f7 service nova] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:22:29.143214 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cbe875-e91f-4894-89ff-3834f37b91f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:22:29.145029 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-d3cbe875-e91f-4894-89ff-3834f37b91f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:22:29.212306 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cbe875-e91f-4894-89ff-3834f37b91f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:22:29.214051 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cbe875-e91f-4894-89ff-3834f37b91f7 service nova] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:22:29.223297 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1160/2316] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:22:29 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:22:33.190492 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e675ed11-f799-4e94-85b7-73cbe6d62fdd None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:22:33.191032 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e675ed11-f799-4e94-85b7-73cbe6d62fdd None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:22:33.191032 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e675ed11-f799-4e94-85b7-73cbe6d62fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:22:33.208626 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-e675ed11-f799-4e94-85b7-73cbe6d62fdd None None] MFA Rules not processed for user `e2e2bd6cf0a44d529ef9b2678c467aba`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:22:33.209847 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e675ed11-f799-4e94-85b7-73cbe6d62fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:22:33.264043 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1157/2317] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:22:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:22:39.220153 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=90 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:22:39.390486 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbe875-e91f-4894-89ff-3834f37b91f7 service nova] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:22:39.404560 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cbe875-e91f-4894-89ff-3834f37b91f7 service nova] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:22:39.410863 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f677e-3250-4a99-8f6c-e00d9c06d662 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:22:39.412094 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f677e-3250-4a99-8f6c-e00d9c06d662 service nova] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:22:39.412376 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f677e-3250-4a99-8f6c-e00d9c06d662 service nova] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:22:39.412646 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f677e-3250-4a99-8f6c-e00d9c06d662 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:22:39.414581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-b09f677e-3250-4a99-8f6c-e00d9c06d662 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:22:39.428281 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09f677e-3250-4a99-8f6c-e00d9c06d662 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:22:39.429934 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09f677e-3250-4a99-8f6c-e00d9c06d662 service nova] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:22:39.441620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1161/2318] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:22:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:22:43.257325 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=53 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:22:44.290993 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e675ed11-f799-4e94-85b7-73cbe6d62fdd None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:22:44.304188 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e675ed11-f799-4e94-85b7-73cbe6d62fdd None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:22:44.309870 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0919a87-e1e5-4fe2-a48f-3179b5f3587b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:22:44.310508 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0919a87-e1e5-4fe2-a48f-3179b5f3587b service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:22:44.310793 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0919a87-e1e5-4fe2-a48f-3179b5f3587b service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:22:44.311077 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0919a87-e1e5-4fe2-a48f-3179b5f3587b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:22:44.312881 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b0919a87-e1e5-4fe2-a48f-3179b5f3587b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:22:44.371978 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0919a87-e1e5-4fe2-a48f-3179b5f3587b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:22:44.373583 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0919a87-e1e5-4fe2-a48f-3179b5f3587b service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:22:44.383683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1158/2319] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:22:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:22:49.436677 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=6 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:22:54.378595 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=28 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:22:54.978029 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b727b-91a4-469b-8264-76e385a4689b None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:22:54.978602 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b727b-91a4-469b-8264-76e385a4689b None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:22:54.978602 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b727b-91a4-469b-8264-76e385a4689b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:22:54.997279 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-340b727b-91a4-469b-8264-76e385a4689b None None] MFA Rules not processed for user `646d135cd569428c8eda3577df418963`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:22:54.998478 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-340b727b-91a4-469b-8264-76e385a4689b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:22:55.091173 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1162/2320] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:22:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:23:00.250346 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45158505-b68f-4cb5-b8eb-be8e49bdb4bc None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:23:00.250831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45158505-b68f-4cb5-b8eb-be8e49bdb4bc None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:23:00.250928 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45158505-b68f-4cb5-b8eb-be8e49bdb4bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:23:00.268495 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-45158505-b68f-4cb5-b8eb-be8e49bdb4bc None None] MFA Rules not processed for user `5e052654361f4580ab36a347610f63c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:23:00.269702 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-45158505-b68f-4cb5-b8eb-be8e49bdb4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:23:00.276985 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1159/2321] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:23:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:23:05.078386 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=25 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:23:10.263713 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:23:26.022284 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b727b-91a4-469b-8264-76e385a4689b None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:23:26.023745 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-340b727b-91a4-469b-8264-76e385a4689b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:23:26.161371 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b727b-91a4-469b-8264-76e385a4689b None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:23:26.167214 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29059336-3886-4dc9-8c3c-4d9637331ddd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64824ba84a024930bbc177f8a8d9a046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:23:26.167896 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29059336-3886-4dc9-8c3c-4d9637331ddd service placement] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:23:26.168181 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29059336-3886-4dc9-8c3c-4d9637331ddd service placement] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:23:26.168453 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29059336-3886-4dc9-8c3c-4d9637331ddd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:23:26.170223 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-29059336-3886-4dc9-8c3c-4d9637331ddd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:23:26.183086 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29059336-3886-4dc9-8c3c-4d9637331ddd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:23:26.184774 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29059336-3886-4dc9-8c3c-4d9637331ddd service placement] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:23:26.195377 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1163/2322] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:23:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:23:36.190659 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=63 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:24:09.320303 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45158505-b68f-4cb5-b8eb-be8e49bdb4bc None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:24:09.321767 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-45158505-b68f-4cb5-b8eb-be8e49bdb4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:24:09.411345 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45158505-b68f-4cb5-b8eb-be8e49bdb4bc None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:24:09.415000 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c849d8-a182-45d3-a86e-c10d09e02b3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:24:09.415669 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c849d8-a182-45d3-a86e-c10d09e02b3f service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:09.415952 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c849d8-a182-45d3-a86e-c10d09e02b3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:09.416257 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c849d8-a182-45d3-a86e-c10d09e02b3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:09.418304 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-50c849d8-a182-45d3-a86e-c10d09e02b3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:24:09.513279 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c849d8-a182-45d3-a86e-c10d09e02b3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:24:09.514750 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c849d8-a182-45d3-a86e-c10d09e02b3f service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:24:09.535292 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1160/2323] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:24:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:24:13.654037 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29059336-3886-4dc9-8c3c-4d9637331ddd service placement] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:24:13.656329 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-29059336-3886-4dc9-8c3c-4d9637331ddd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:24:13.672431 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29059336-3886-4dc9-8c3c-4d9637331ddd service placement] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:24:13.679106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c522c5a-098a-4bf8-9c57-a4d95ac2c495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:24:13.679924 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c522c5a-098a-4bf8-9c57-a4d95ac2c495 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:13.680223 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c522c5a-098a-4bf8-9c57-a4d95ac2c495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:13.680487 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c522c5a-098a-4bf8-9c57-a4d95ac2c495 admin admin] PATH_INFO: `/v3/users/e2e2bd6cf0a44d529ef9b2678c467aba` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:13.681797 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c522c5a-098a-4bf8-9c57-a4d95ac2c495 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2e2bd6cf0a44d529ef9b2678c467aba)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:24:13.684122 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c522c5a-098a-4bf8-9c57-a4d95ac2c495 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:24:13.746129 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-8c522c5a-098a-4bf8-9c57-a4d95ac2c495 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2e2bd6cf0a44d529ef9b2678c467aba'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:13.747203 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-8c522c5a-098a-4bf8-9c57-a4d95ac2c495 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2e2bd6cf0a44d529ef9b2678c467aba'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:13.763812 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-8c522c5a-098a-4bf8-9c57-a4d95ac2c495 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2e2bd6cf0a44d529ef9b2678c467aba'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:13.771473 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-8c522c5a-098a-4bf8-9c57-a4d95ac2c495 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2e2bd6cf0a44d529ef9b2678c467aba'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:13.772246 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-8c522c5a-098a-4bf8-9c57-a4d95ac2c495 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2e2bd6cf0a44d529ef9b2678c467aba'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:13.796877 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1164/2324] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:24:13 2022] DELETE /identity/v3/users/e2e2bd6cf0a44d529ef9b2678c467aba => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:24:14.125941 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c849d8-a182-45d3-a86e-c10d09e02b3f service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:24:14.140110 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c849d8-a182-45d3-a86e-c10d09e02b3f service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:24:14.205738 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:24:14.224481 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f0b32-af54-4a89-9abe-01e05a08becf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:24:14.225360 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f0b32-af54-4a89-9abe-01e05a08becf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:14.225634 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f0b32-af54-4a89-9abe-01e05a08becf admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:14.225910 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f0b32-af54-4a89-9abe-01e05a08becf admin admin] PATH_INFO: `/v3/projects/dc01ed4482c04899b1509a655c9c7e3a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:14.227152 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3f0b32-af54-4a89-9abe-01e05a08becf admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc01ed4482c04899b1509a655c9c7e3a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:24:14.229108 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3f0b32-af54-4a89-9abe-01e05a08becf admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:24:14.284262 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0a3f0b32-af54-4a89-9abe-01e05a08becf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc01ed4482c04899b1509a655c9c7e3a', 'request_id': 'req-0a3f0b32-af54-4a89-9abe-01e05a08becf'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:14.291566 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0a3f0b32-af54-4a89-9abe-01e05a08becf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc01ed4482c04899b1509a655c9c7e3a', 'request_id': 'req-0a3f0b32-af54-4a89-9abe-01e05a08becf'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:14.303974 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0a3f0b32-af54-4a89-9abe-01e05a08becf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc01ed4482c04899b1509a655c9c7e3a', 'request_id': 'req-0a3f0b32-af54-4a89-9abe-01e05a08becf'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:14.319923 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0a3f0b32-af54-4a89-9abe-01e05a08becf admin admin] The token cache is being invalidate because project dc01ed4482c04899b1509a655c9c7e3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:24:14.320428 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0a3f0b32-af54-4a89-9abe-01e05a08becf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:14.321161 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0a3f0b32-af54-4a89-9abe-01e05a08becf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:14.323564 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1161/2325] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:24:14 2022] DELETE /identity/v3/projects/dc01ed4482c04899b1509a655c9c7e3a => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:24:14.329360 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbb9ec-0a86-4bae-9afb-5936d73fc28d None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:14.329796 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbb9ec-0a86-4bae-9afb-5936d73fc28d None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:14.330225 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbb9ec-0a86-4bae-9afb-5936d73fc28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:14.443472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-a2dbb9ec-0a86-4bae-9afb-5936d73fc28d None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:24:14.445344 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-a2dbb9ec-0a86-4bae-9afb-5936d73fc28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:24:14.542574 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1165/2326] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:24:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:24:14.555828 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05826c41-98d1-46b9-b688-a4bea5d5cf17 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:14.556122 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05826c41-98d1-46b9-b688-a4bea5d5cf17 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:14.559280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05826c41-98d1-46b9-b688-a4bea5d5cf17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:14.584644 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-05826c41-98d1-46b9-b688-a4bea5d5cf17 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:24:14.586228 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-05826c41-98d1-46b9-b688-a4bea5d5cf17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:24:14.601680 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1162/2327] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:24:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:24:14.601990 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbb9ec-0a86-4bae-9afb-5936d73fc28d None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:24:14.614805 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbb9ec-0a86-4bae-9afb-5936d73fc28d None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:24:14.622060 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5eeb59-4414-47ec-84c1-3e9359e2f0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:24:14.622851 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5eeb59-4414-47ec-84c1-3e9359e2f0d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:14.623213 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5eeb59-4414-47ec-84c1-3e9359e2f0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:14.623612 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5eeb59-4414-47ec-84c1-3e9359e2f0d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:14.624464 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5eeb59-4414-47ec-84c1-3e9359e2f0d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:24:14.624842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5eeb59-4414-47ec-84c1-3e9359e2f0d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:24:14.626642 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5eeb59-4414-47ec-84c1-3e9359e2f0d5 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:24:14.646507 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1166/2328] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:24:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:24:14.654397 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05826c41-98d1-46b9-b688-a4bea5d5cf17 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:24:14.677892 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05826c41-98d1-46b9-b688-a4bea5d5cf17 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:24:14.679994 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af33d78-fb74-46a9-b1c4-72d5117302f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:24:14.680838 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af33d78-fb74-46a9-b1c4-72d5117302f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:14.681227 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af33d78-fb74-46a9-b1c4-72d5117302f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:14.681509 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af33d78-fb74-46a9-b1c4-72d5117302f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:14.682928 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af33d78-fb74-46a9-b1c4-72d5117302f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:24:14.684938 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af33d78-fb74-46a9-b1c4-72d5117302f9 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:24:14.712847 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1163/2329] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:24:14 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:24:14.717053 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5eeb59-4414-47ec-84c1-3e9359e2f0d5 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:24:14.727767 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5eeb59-4414-47ec-84c1-3e9359e2f0d5 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:24:14.755779 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:24:14.797492 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f4ae28-6116-4afd-ba98-c16bf20f261e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:24:14.798361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4ae28-6116-4afd-ba98-c16bf20f261e admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:14.798718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4ae28-6116-4afd-ba98-c16bf20f261e admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:14.799061 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4ae28-6116-4afd-ba98-c16bf20f261e admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:14.800047 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f4ae28-6116-4afd-ba98-c16bf20f261e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:24:14.802264 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f4ae28-6116-4afd-ba98-c16bf20f261e admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:24:14.825192 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1167/2330] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:24:14 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:24:14.829419 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af33d78-fb74-46a9-b1c4-72d5117302f9 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:24:14.840504 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af33d78-fb74-46a9-b1c4-72d5117302f9 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:24:14.845616 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88352761-6142-4728-83ba-473a7e581b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:24:14.846983 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88352761-6142-4728-83ba-473a7e581b24 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:14.846983 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88352761-6142-4728-83ba-473a7e581b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:14.846983 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88352761-6142-4728-83ba-473a7e581b24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:14.847633 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88352761-6142-4728-83ba-473a7e581b24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:24:14.849274 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88352761-6142-4728-83ba-473a7e581b24 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:24:14.864518 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1164/2331] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:24:14.868701 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f4ae28-6116-4afd-ba98-c16bf20f261e admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:24:14.880427 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f4ae28-6116-4afd-ba98-c16bf20f261e admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:24:14.885770 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3284f-b40c-4f9d-b68d-36656d20b4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:24:14.886465 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3284f-b40c-4f9d-b68d-36656d20b4d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:14.886752 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3284f-b40c-4f9d-b68d-36656d20b4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:14.887032 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3284f-b40c-4f9d-b68d-36656d20b4d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:14.887819 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d3284f-b40c-4f9d-b68d-36656d20b4d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:24:14.889470 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d3284f-b40c-4f9d-b68d-36656d20b4d6 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:24:14.900880 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1168/2332] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:24:14.905896 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88352761-6142-4728-83ba-473a7e581b24 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:24:14.918298 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88352761-6142-4728-83ba-473a7e581b24 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:24:14.924612 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87181275-d54a-4a97-9803-4cdf9e6c394b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:24:14.925786 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87181275-d54a-4a97-9803-4cdf9e6c394b admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:14.926283 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87181275-d54a-4a97-9803-4cdf9e6c394b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:14.926675 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87181275-d54a-4a97-9803-4cdf9e6c394b admin admin] PATH_INFO: `/v3/projects/333ace0873e1456fa3aa63c213ec3b95/users/0560078127e44d42a299d5515b038a21/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:14.945365 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87181275-d54a-4a97-9803-4cdf9e6c394b admin admin] RBAC: Authorizing `identity:create_grant(project_id=333ace0873e1456fa3aa63c213ec3b95, user_id=0560078127e44d42a299d5515b038a21, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:24:14.948264 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87181275-d54a-4a97-9803-4cdf9e6c394b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:24:14.964435 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1165/2333] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:24:14 2022] PUT /identity/v3/projects/333ace0873e1456fa3aa63c213ec3b95/users/0560078127e44d42a299d5515b038a21/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:24:14.975140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50a33f-ef7a-4460-bb7f-57b0d5f5e4fc None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:14.975464 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50a33f-ef7a-4460-bb7f-57b0d5f5e4fc None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:14.975901 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50a33f-ef7a-4460-bb7f-57b0d5f5e4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:15.006246 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-9e50a33f-ef7a-4460-bb7f-57b0d5f5e4fc None None] MFA Rules not processed for user `0560078127e44d42a299d5515b038a21`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:24:15.007506 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-9e50a33f-ef7a-4460-bb7f-57b0d5f5e4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:24:15.073903 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1169/2334] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:24:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:24:15.085782 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87181275-d54a-4a97-9803-4cdf9e6c394b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:24:15.087624 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-87181275-d54a-4a97-9803-4cdf9e6c394b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:24:15.172845 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:24:15.204997 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87181275-d54a-4a97-9803-4cdf9e6c394b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:24:15.211057 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c5c605-48b5-4a37-97cf-e90815e8160e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:24:15.211747 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5c605-48b5-4a37-97cf-e90815e8160e service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:15.212037 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5c605-48b5-4a37-97cf-e90815e8160e service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:15.212318 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5c605-48b5-4a37-97cf-e90815e8160e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:15.221296 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c5c605-48b5-4a37-97cf-e90815e8160e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:24:15.222894 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c5c605-48b5-4a37-97cf-e90815e8160e service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:24:15.232744 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1166/2335] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:24:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:24:17.658187 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2a6c0-c977-48ae-b4d1-d8d761d7d3d7 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:17.658736 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2a6c0-c977-48ae-b4d1-d8d761d7d3d7 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:17.658736 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2a6c0-c977-48ae-b4d1-d8d761d7d3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:17.676229 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-c3d2a6c0-c977-48ae-b4d1-d8d761d7d3d7 None None] MFA Rules not processed for user `75652cd3bfa04c7ebae2cbea77cb5fd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:24:17.677435 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-c3d2a6c0-c977-48ae-b4d1-d8d761d7d3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:24:17.731326 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1170/2336] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:24:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:24:24.646949 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c5c605-48b5-4a37-97cf-e90815e8160e service nova] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:24:24.722920 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c5c605-48b5-4a37-97cf-e90815e8160e service nova] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:24:24.727896 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d6cc8-c146-4327-a4b1-2f743f3aef51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:24:24.728812 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d6cc8-c146-4327-a4b1-2f743f3aef51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:24.729089 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d6cc8-c146-4327-a4b1-2f743f3aef51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:24.729385 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d6cc8-c146-4327-a4b1-2f743f3aef51 admin admin] PATH_INFO: `/v3/users/0560078127e44d42a299d5515b038a21` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:24.730622 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d6cc8-c146-4327-a4b1-2f743f3aef51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0560078127e44d42a299d5515b038a21)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:24:24.732636 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d6cc8-c146-4327-a4b1-2f743f3aef51 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:24:24.790168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-9d0d6cc8-c146-4327-a4b1-2f743f3aef51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0560078127e44d42a299d5515b038a21'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:24.790921 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-9d0d6cc8-c146-4327-a4b1-2f743f3aef51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0560078127e44d42a299d5515b038a21'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:24.803683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-9d0d6cc8-c146-4327-a4b1-2f743f3aef51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0560078127e44d42a299d5515b038a21'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:24.811319 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-9d0d6cc8-c146-4327-a4b1-2f743f3aef51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0560078127e44d42a299d5515b038a21'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:24.819527 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-9d0d6cc8-c146-4327-a4b1-2f743f3aef51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0560078127e44d42a299d5515b038a21'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:24.826400 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1167/2337] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:24:24 2022] DELETE /identity/v3/users/0560078127e44d42a299d5515b038a21 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:24:24.831002 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4acd96f-fd7c-486c-abcc-a827a5022b26 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:24.831498 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4acd96f-fd7c-486c-abcc-a827a5022b26 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:24.831654 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4acd96f-fd7c-486c-abcc-a827a5022b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:24.850134 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-a4acd96f-fd7c-486c-abcc-a827a5022b26 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:24:24.851315 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-a4acd96f-fd7c-486c-abcc-a827a5022b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:24:24.893746 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:24:24.917085 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1171/2338] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:24:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:24:24.927669 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d6cc8-c146-4327-a4b1-2f743f3aef51 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:24:24.929262 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9d0d6cc8-c146-4327-a4b1-2f743f3aef51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:24:24.996679 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d6cc8-c146-4327-a4b1-2f743f3aef51 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:24:25.000774 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528eea14-d10a-4d15-b7b4-a6b6031df877 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:24:25.001420 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528eea14-d10a-4d15-b7b4-a6b6031df877 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:25.001700 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528eea14-d10a-4d15-b7b4-a6b6031df877 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:25.001982 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528eea14-d10a-4d15-b7b4-a6b6031df877 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:25.009520 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:24:25.012980 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528eea14-d10a-4d15-b7b4-a6b6031df877 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:24:25.014959 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528eea14-d10a-4d15-b7b4-a6b6031df877 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:24:25.026686 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1168/2339] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:24:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:24:25.403110 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4acd96f-fd7c-486c-abcc-a827a5022b26 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:24:25.406104 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-a4acd96f-fd7c-486c-abcc-a827a5022b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:24:25.419901 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4acd96f-fd7c-486c-abcc-a827a5022b26 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:24:25.424650 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9fe7a3-d012-4bdc-82bc-b596903693df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:24:25.425677 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9fe7a3-d012-4bdc-82bc-b596903693df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:25.425972 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9fe7a3-d012-4bdc-82bc-b596903693df admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:25.426254 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9fe7a3-d012-4bdc-82bc-b596903693df admin admin] PATH_INFO: `/v3/projects/333ace0873e1456fa3aa63c213ec3b95` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:25.427476 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9fe7a3-d012-4bdc-82bc-b596903693df admin admin] RBAC: Authorizing `identity:delete_project(project_id=333ace0873e1456fa3aa63c213ec3b95)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:24:25.429453 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9fe7a3-d012-4bdc-82bc-b596903693df admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:24:25.479567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-ca9fe7a3-d012-4bdc-82bc-b596903693df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '333ace0873e1456fa3aa63c213ec3b95', 'request_id': 'req-ca9fe7a3-d012-4bdc-82bc-b596903693df'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:25.486017 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-ca9fe7a3-d012-4bdc-82bc-b596903693df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '333ace0873e1456fa3aa63c213ec3b95', 'request_id': 'req-ca9fe7a3-d012-4bdc-82bc-b596903693df'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:25.494236 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-ca9fe7a3-d012-4bdc-82bc-b596903693df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '333ace0873e1456fa3aa63c213ec3b95', 'request_id': 'req-ca9fe7a3-d012-4bdc-82bc-b596903693df'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:25.504984 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-ca9fe7a3-d012-4bdc-82bc-b596903693df admin admin] The token cache is being invalidate because project 333ace0873e1456fa3aa63c213ec3b95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:24:25.505276 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-ca9fe7a3-d012-4bdc-82bc-b596903693df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:25.505920 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-ca9fe7a3-d012-4bdc-82bc-b596903693df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:25.507487 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1172/2340] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:24:25 2022] DELETE /identity/v3/projects/333ace0873e1456fa3aa63c213ec3b95 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:24:25.512954 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0270be-d006-413a-aac5-3871b409fab6 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:25.513245 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0270be-d006-413a-aac5-3871b409fab6 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:25.513527 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0270be-d006-413a-aac5-3871b409fab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:25.532098 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-7a0270be-d006-413a-aac5-3871b409fab6 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:24:25.533723 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7a0270be-d006-413a-aac5-3871b409fab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:24:25.597189 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1169/2341] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:24:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:24:25.605369 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43b754-c7cd-436c-b451-6beb69bf1441 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:25.605651 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43b754-c7cd-436c-b451-6beb69bf1441 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:25.605928 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43b754-c7cd-436c-b451-6beb69bf1441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:25.624703 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-ad43b754-c7cd-436c-b451-6beb69bf1441 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:24:25.626058 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-ad43b754-c7cd-436c-b451-6beb69bf1441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:24:25.634624 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1173/2342] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:24:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:24:25.638990 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0270be-d006-413a-aac5-3871b409fab6 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:24:25.648831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0270be-d006-413a-aac5-3871b409fab6 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:24:25.654169 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723cde8b-970b-494a-b939-608fa848a320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:24:25.654837 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cde8b-970b-494a-b939-608fa848a320 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:25.655121 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cde8b-970b-494a-b939-608fa848a320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:25.655396 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cde8b-970b-494a-b939-608fa848a320 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:25.656170 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723cde8b-970b-494a-b939-608fa848a320 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:24:25.656484 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723cde8b-970b-494a-b939-608fa848a320 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:24:25.658103 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723cde8b-970b-494a-b939-608fa848a320 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:24:25.672337 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1170/2343] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:24:25.676832 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43b754-c7cd-436c-b451-6beb69bf1441 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:24:25.687135 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43b754-c7cd-436c-b451-6beb69bf1441 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:24:25.692046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d1053-a74d-46a7-8ae0-9590af52756e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:24:25.692713 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d1053-a74d-46a7-8ae0-9590af52756e admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:25.692990 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d1053-a74d-46a7-8ae0-9590af52756e admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:25.693286 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d1053-a74d-46a7-8ae0-9590af52756e admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:25.693962 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d1053-a74d-46a7-8ae0-9590af52756e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:24:25.695919 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d1053-a74d-46a7-8ae0-9590af52756e admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:24:25.738064 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1174/2344] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:24:25 2022] POST /identity/v3/projects => generated 405 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:24:25.742880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723cde8b-970b-494a-b939-608fa848a320 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:24:25.760870 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723cde8b-970b-494a-b939-608fa848a320 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:24:25.857076 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a3ab3-53e4-4639-99b2-6ec11edfcbdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:24:25.857746 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a3ab3-53e4-4639-99b2-6ec11edfcbdc admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:25.858033 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a3ab3-53e4-4639-99b2-6ec11edfcbdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:25.858309 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a3ab3-53e4-4639-99b2-6ec11edfcbdc admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:25.859008 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376a3ab3-53e4-4639-99b2-6ec11edfcbdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:24:25.861010 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376a3ab3-53e4-4639-99b2-6ec11edfcbdc admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:24:25.882899 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1171/2345] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:24:25 2022] POST /identity/v3/users => generated 303 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:24:25.889327 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d1053-a74d-46a7-8ae0-9590af52756e admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:24:25.901229 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d1053-a74d-46a7-8ae0-9590af52756e admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:24:25.907037 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189bbab2-7b04-40cb-b794-478dcfb7b465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:24:25.908391 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189bbab2-7b04-40cb-b794-478dcfb7b465 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:25.908391 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189bbab2-7b04-40cb-b794-478dcfb7b465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:25.908391 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189bbab2-7b04-40cb-b794-478dcfb7b465 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:25.909140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189bbab2-7b04-40cb-b794-478dcfb7b465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:24:25.910689 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189bbab2-7b04-40cb-b794-478dcfb7b465 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:24:25.924476 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1175/2346] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:24:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:24:25.928417 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a3ab3-53e4-4639-99b2-6ec11edfcbdc admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:24:25.940315 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a3ab3-53e4-4639-99b2-6ec11edfcbdc admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:24:25.949264 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f92351-6ff7-4ef4-af03-70d549611c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:24:25.950182 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f92351-6ff7-4ef4-af03-70d549611c2f admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:25.950421 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f92351-6ff7-4ef4-af03-70d549611c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:25.950619 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f92351-6ff7-4ef4-af03-70d549611c2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:25.951362 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f92351-6ff7-4ef4-af03-70d549611c2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:24:25.953071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f92351-6ff7-4ef4-af03-70d549611c2f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:24:25.973148 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1172/2347] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:24:25 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:24:25.976934 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189bbab2-7b04-40cb-b794-478dcfb7b465 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:24:25.987680 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189bbab2-7b04-40cb-b794-478dcfb7b465 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:24:25.993580 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ef2d4-656e-4933-87ee-c0540a37475a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:24:25.994538 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ef2d4-656e-4933-87ee-c0540a37475a admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:25.994823 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ef2d4-656e-4933-87ee-c0540a37475a admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:25.995103 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ef2d4-656e-4933-87ee-c0540a37475a admin admin] PATH_INFO: `/v3/projects/1e6275b350cc4f939fba7eb6bf765dd4/users/e24860e2e1e84a7a94d2008d56fa8612/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:26.012722 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846ef2d4-656e-4933-87ee-c0540a37475a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e6275b350cc4f939fba7eb6bf765dd4, user_id=e24860e2e1e84a7a94d2008d56fa8612, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:24:26.015692 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846ef2d4-656e-4933-87ee-c0540a37475a admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:24:26.039999 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1176/2348] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:24:25 2022] PUT /identity/v3/projects/1e6275b350cc4f939fba7eb6bf765dd4/users/e24860e2e1e84a7a94d2008d56fa8612/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:24:26.045251 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a2a36-3daf-4733-beed-fe3705ec85e4 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:26.045251 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a2a36-3daf-4733-beed-fe3705ec85e4 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:26.045650 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a2a36-3daf-4733-beed-fe3705ec85e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:26.075981 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-0a9a2a36-3daf-4733-beed-fe3705ec85e4 None None] MFA Rules not processed for user `e24860e2e1e84a7a94d2008d56fa8612`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:24:26.077243 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0a9a2a36-3daf-4733-beed-fe3705ec85e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:24:26.131523 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:24:26.154625 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1173/2349] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:24:26 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:24:26.159179 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ef2d4-656e-4933-87ee-c0540a37475a admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:24:26.225275 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ef2d4-656e-4933-87ee-c0540a37475a admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:24:26.230192 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da869b2f-07b7-4272-9875-813123e5caf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:24:26.230856 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da869b2f-07b7-4272-9875-813123e5caf0 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:26.231137 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da869b2f-07b7-4272-9875-813123e5caf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:26.231414 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da869b2f-07b7-4272-9875-813123e5caf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:26.232138 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da869b2f-07b7-4272-9875-813123e5caf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:24:26.234069 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da869b2f-07b7-4272-9875-813123e5caf0 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:24:26.238404 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:24:26.267924 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1177/2350] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:24:26 2022] POST /identity/v3/projects => generated 407 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:24:26.269193 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a2a36-3daf-4733-beed-fe3705ec85e4 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:24:26.280482 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a2a36-3daf-4733-beed-fe3705ec85e4 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:24:26.350293 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d3363f-6c43-407f-8c80-b206a54c0562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:24:26.351145 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d3363f-6c43-407f-8c80-b206a54c0562 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:26.351430 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d3363f-6c43-407f-8c80-b206a54c0562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:26.351737 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d3363f-6c43-407f-8c80-b206a54c0562 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:26.352431 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d3363f-6c43-407f-8c80-b206a54c0562 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:24:26.358969 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d3363f-6c43-407f-8c80-b206a54c0562 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:24:26.384659 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1174/2351] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:24:26 2022] POST /identity/v3/users => generated 310 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:24:26.389565 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da869b2f-07b7-4272-9875-813123e5caf0 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:24:26.400746 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da869b2f-07b7-4272-9875-813123e5caf0 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:24:26.407683 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d722c4bd-91b5-4453-b2f9-4cbe23c34a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:24:26.408494 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722c4bd-91b5-4453-b2f9-4cbe23c34a31 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:26.408818 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722c4bd-91b5-4453-b2f9-4cbe23c34a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:26.409197 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722c4bd-91b5-4453-b2f9-4cbe23c34a31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:26.410270 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d722c4bd-91b5-4453-b2f9-4cbe23c34a31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:24:26.412227 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d722c4bd-91b5-4453-b2f9-4cbe23c34a31 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:24:26.425041 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1178/2352] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:24:26.429273 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d3363f-6c43-407f-8c80-b206a54c0562 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:24:26.441536 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d3363f-6c43-407f-8c80-b206a54c0562 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:24:26.447112 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca046570-d11c-44e7-8093-923e063de09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:24:26.448713 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca046570-d11c-44e7-8093-923e063de09f admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:26.449213 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca046570-d11c-44e7-8093-923e063de09f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:26.449483 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca046570-d11c-44e7-8093-923e063de09f admin admin] PATH_INFO: `/v3/projects/15beef36ea194a5e889f91d4478fcf24/users/19dc882d86c840be903d11920ebe81d0/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:26.467820 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca046570-d11c-44e7-8093-923e063de09f admin admin] RBAC: Authorizing `identity:create_grant(project_id=15beef36ea194a5e889f91d4478fcf24, user_id=19dc882d86c840be903d11920ebe81d0, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:24:26.471814 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca046570-d11c-44e7-8093-923e063de09f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:24:26.494698 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1175/2353] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:24:26 2022] PUT /identity/v3/projects/15beef36ea194a5e889f91d4478fcf24/users/19dc882d86c840be903d11920ebe81d0/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:24:26.496256 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d722c4bd-91b5-4453-b2f9-4cbe23c34a31 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:24:26.507615 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d722c4bd-91b5-4453-b2f9-4cbe23c34a31 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:24:26.573516 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a8024-1936-45ae-9b54-4b73ee225990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:24:26.574287 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a8024-1936-45ae-9b54-4b73ee225990 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:26.574570 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a8024-1936-45ae-9b54-4b73ee225990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:26.574842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a8024-1936-45ae-9b54-4b73ee225990 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:26.575602 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a8024-1936-45ae-9b54-4b73ee225990 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:24:26.577228 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a8024-1936-45ae-9b54-4b73ee225990 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:24:26.588046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1179/2354] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:24:26.592699 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca046570-d11c-44e7-8093-923e063de09f admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:24:26.603423 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca046570-d11c-44e7-8093-923e063de09f admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:24:26.609621 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779cd60-119f-4a36-b0de-bac3c8f4e74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:24:26.610525 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779cd60-119f-4a36-b0de-bac3c8f4e74f admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:26.610802 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779cd60-119f-4a36-b0de-bac3c8f4e74f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:26.611078 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779cd60-119f-4a36-b0de-bac3c8f4e74f admin admin] PATH_INFO: `/v3/projects/15beef36ea194a5e889f91d4478fcf24/users/19dc882d86c840be903d11920ebe81d0/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:26.613140 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2779cd60-119f-4a36-b0de-bac3c8f4e74f admin admin] RBAC: Authorizing `identity:create_grant(project_id=15beef36ea194a5e889f91d4478fcf24, user_id=19dc882d86c840be903d11920ebe81d0, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:24:26.615846 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2779cd60-119f-4a36-b0de-bac3c8f4e74f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:24:26.628734 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1176/2355] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:24:26 2022] PUT /identity/v3/projects/15beef36ea194a5e889f91d4478fcf24/users/19dc882d86c840be903d11920ebe81d0/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:24:26.640079 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449521f-945e-4afb-add0-ceb2cd97332f None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:26.640510 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449521f-945e-4afb-add0-ceb2cd97332f None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:26.640888 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449521f-945e-4afb-add0-ceb2cd97332f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:26.684911 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-8449521f-945e-4afb-add0-ceb2cd97332f None None] MFA Rules not processed for user `19dc882d86c840be903d11920ebe81d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:24:26.687106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-8449521f-945e-4afb-add0-ceb2cd97332f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:24:26.795694 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1180/2356] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:24:26 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:24:26.807735 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779cd60-119f-4a36-b0de-bac3c8f4e74f admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:24:26.809574 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-2779cd60-119f-4a36-b0de-bac3c8f4e74f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:24:26.939789 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779cd60-119f-4a36-b0de-bac3c8f4e74f admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:24:26.943989 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10bfa2e-9f10-4909-a1ce-2db3237fefdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:24:26.944663 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bfa2e-9f10-4909-a1ce-2db3237fefdd service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:26.944951 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bfa2e-9f10-4909-a1ce-2db3237fefdd service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:26.945241 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bfa2e-9f10-4909-a1ce-2db3237fefdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:26.956149 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10bfa2e-9f10-4909-a1ce-2db3237fefdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:24:26.957668 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10bfa2e-9f10-4909-a1ce-2db3237fefdd service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:24:26.969406 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1177/2357] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:24:26 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:24:27.192220 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449521f-945e-4afb-add0-ceb2cd97332f None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:24:27.202921 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449521f-945e-4afb-add0-ceb2cd97332f None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:24:27.209947 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddaf100-3125-406e-a499-3ae107e61704 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:24:27.210664 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddaf100-3125-406e-a499-3ae107e61704 service cinder] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:27.211013 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddaf100-3125-406e-a499-3ae107e61704 service cinder] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:27.211294 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddaf100-3125-406e-a499-3ae107e61704 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:27.274260 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:24:27.277875 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddaf100-3125-406e-a499-3ae107e61704 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:24:27.279464 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddaf100-3125-406e-a499-3ae107e61704 service cinder] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:24:27.290833 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1181/2358] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:24:27 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:24:36.964304 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=83,keystone:INSERT=5 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:24:37.140360 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10bfa2e-9f10-4909-a1ce-2db3237fefdd service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:24:37.222428 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10bfa2e-9f10-4909-a1ce-2db3237fefdd service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:24:37.233656 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc89346-8795-4da1-b9b3-4395d2746dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:24:37.234553 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc89346-8795-4da1-b9b3-4395d2746dc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:37.234833 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc89346-8795-4da1-b9b3-4395d2746dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:37.235107 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc89346-8795-4da1-b9b3-4395d2746dc1 admin admin] PATH_INFO: `/v3/users/e24860e2e1e84a7a94d2008d56fa8612` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:37.236512 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc89346-8795-4da1-b9b3-4395d2746dc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e24860e2e1e84a7a94d2008d56fa8612)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:24:37.238478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc89346-8795-4da1-b9b3-4395d2746dc1 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:24:37.289578 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=2 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:24:37.302783 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-5bc89346-8795-4da1-b9b3-4395d2746dc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e24860e2e1e84a7a94d2008d56fa8612'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:37.303870 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-5bc89346-8795-4da1-b9b3-4395d2746dc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e24860e2e1e84a7a94d2008d56fa8612'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:37.327645 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-5bc89346-8795-4da1-b9b3-4395d2746dc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e24860e2e1e84a7a94d2008d56fa8612'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:37.339450 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-5bc89346-8795-4da1-b9b3-4395d2746dc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e24860e2e1e84a7a94d2008d56fa8612'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:37.350787 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-5bc89346-8795-4da1-b9b3-4395d2746dc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e24860e2e1e84a7a94d2008d56fa8612'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:37.365265 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1178/2359] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:24:37 2022] DELETE /identity/v3/users/e24860e2e1e84a7a94d2008d56fa8612 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:24:37.368391 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164634d8-3fb9-46a3-84b4-2a1b43854a9c None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:37.368391 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164634d8-3fb9-46a3-84b4-2a1b43854a9c None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:37.368818 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164634d8-3fb9-46a3-84b4-2a1b43854a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:37.391618 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-164634d8-3fb9-46a3-84b4-2a1b43854a9c None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:24:37.392850 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-164634d8-3fb9-46a3-84b4-2a1b43854a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:24:37.483525 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1182/2360] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:24:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:24:37.493903 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc89346-8795-4da1-b9b3-4395d2746dc1 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:24:37.495359 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5bc89346-8795-4da1-b9b3-4395d2746dc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:24:37.578302 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc89346-8795-4da1-b9b3-4395d2746dc1 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:24:37.582262 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9953aad5-b6d9-42d6-9ccf-3c2b9c2f8ae8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:24:37.582937 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953aad5-b6d9-42d6-9ccf-3c2b9c2f8ae8 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:37.583220 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953aad5-b6d9-42d6-9ccf-3c2b9c2f8ae8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:37.583501 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953aad5-b6d9-42d6-9ccf-3c2b9c2f8ae8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:37.594908 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9953aad5-b6d9-42d6-9ccf-3c2b9c2f8ae8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:24:37.596419 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9953aad5-b6d9-42d6-9ccf-3c2b9c2f8ae8 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:24:37.608432 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1179/2361] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:24:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:24:37.897003 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164634d8-3fb9-46a3-84b4-2a1b43854a9c None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:24:37.898461 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-164634d8-3fb9-46a3-84b4-2a1b43854a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:24:37.914796 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164634d8-3fb9-46a3-84b4-2a1b43854a9c None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:24:37.922146 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ba28a-53a4-4fd1-a6f2-0b31a25e10aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:24:37.924078 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ba28a-53a4-4fd1-a6f2-0b31a25e10aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:37.924371 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ba28a-53a4-4fd1-a6f2-0b31a25e10aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:37.924647 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ba28a-53a4-4fd1-a6f2-0b31a25e10aa admin admin] PATH_INFO: `/v3/projects/1e6275b350cc4f939fba7eb6bf765dd4` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:37.926238 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ba28a-53a4-4fd1-a6f2-0b31a25e10aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e6275b350cc4f939fba7eb6bf765dd4)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:24:37.928326 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ba28a-53a4-4fd1-a6f2-0b31a25e10aa admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:24:37.984760 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-562ba28a-53a4-4fd1-a6f2-0b31a25e10aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e6275b350cc4f939fba7eb6bf765dd4', 'request_id': 'req-562ba28a-53a4-4fd1-a6f2-0b31a25e10aa'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:37.992142 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-562ba28a-53a4-4fd1-a6f2-0b31a25e10aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e6275b350cc4f939fba7eb6bf765dd4', 'request_id': 'req-562ba28a-53a4-4fd1-a6f2-0b31a25e10aa'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:38.001195 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-562ba28a-53a4-4fd1-a6f2-0b31a25e10aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e6275b350cc4f939fba7eb6bf765dd4', 'request_id': 'req-562ba28a-53a4-4fd1-a6f2-0b31a25e10aa'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:38.019883 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-562ba28a-53a4-4fd1-a6f2-0b31a25e10aa admin admin] The token cache is being invalidate because project 1e6275b350cc4f939fba7eb6bf765dd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:24:38.020131 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-562ba28a-53a4-4fd1-a6f2-0b31a25e10aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:38.023310 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-562ba28a-53a4-4fd1-a6f2-0b31a25e10aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:38.023310 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1183/2362] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:24:37 2022] DELETE /identity/v3/projects/1e6275b350cc4f939fba7eb6bf765dd4 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:24:38.027942 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9953aad5-b6d9-42d6-9ccf-3c2b9c2f8ae8 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:24:38.037871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9953aad5-b6d9-42d6-9ccf-3c2b9c2f8ae8 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:24:38.059271 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:24:38.113080 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f2a05-e9d7-4306-969e-dfdf45ef009e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:24:38.113990 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f2a05-e9d7-4306-969e-dfdf45ef009e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:38.114301 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f2a05-e9d7-4306-969e-dfdf45ef009e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:38.114609 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f2a05-e9d7-4306-969e-dfdf45ef009e admin admin] PATH_INFO: `/v3/users/19dc882d86c840be903d11920ebe81d0` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:38.116013 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525f2a05-e9d7-4306-969e-dfdf45ef009e admin admin] RBAC: Authorizing `identity:delete_user(user_id=19dc882d86c840be903d11920ebe81d0)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:24:38.118106 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525f2a05-e9d7-4306-969e-dfdf45ef009e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:24:38.167289 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-525f2a05-e9d7-4306-969e-dfdf45ef009e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19dc882d86c840be903d11920ebe81d0'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:38.168232 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-525f2a05-e9d7-4306-969e-dfdf45ef009e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19dc882d86c840be903d11920ebe81d0'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:38.180610 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-525f2a05-e9d7-4306-969e-dfdf45ef009e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19dc882d86c840be903d11920ebe81d0'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:38.188201 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-525f2a05-e9d7-4306-969e-dfdf45ef009e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19dc882d86c840be903d11920ebe81d0'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:38.196907 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-525f2a05-e9d7-4306-969e-dfdf45ef009e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19dc882d86c840be903d11920ebe81d0'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:38.206329 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1180/2363] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:24:38 2022] DELETE /identity/v3/users/19dc882d86c840be903d11920ebe81d0 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:24:38.505119 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ba28a-53a4-4fd1-a6f2-0b31a25e10aa admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:24:38.507331 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-562ba28a-53a4-4fd1-a6f2-0b31a25e10aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:24:38.529093 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ba28a-53a4-4fd1-a6f2-0b31a25e10aa admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:24:38.609025 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5354d415-2565-425e-bdd2-66c4b594a950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:24:38.609860 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354d415-2565-425e-bdd2-66c4b594a950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:38.610137 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354d415-2565-425e-bdd2-66c4b594a950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:38.610408 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354d415-2565-425e-bdd2-66c4b594a950 admin admin] PATH_INFO: `/v3/projects/15beef36ea194a5e889f91d4478fcf24` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:38.611653 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5354d415-2565-425e-bdd2-66c4b594a950 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15beef36ea194a5e889f91d4478fcf24)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:24:38.613635 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5354d415-2565-425e-bdd2-66c4b594a950 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:24:38.646770 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:24:38.667924 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-5354d415-2565-425e-bdd2-66c4b594a950 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15beef36ea194a5e889f91d4478fcf24', 'request_id': 'req-5354d415-2565-425e-bdd2-66c4b594a950'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:38.674932 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-5354d415-2565-425e-bdd2-66c4b594a950 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15beef36ea194a5e889f91d4478fcf24', 'request_id': 'req-5354d415-2565-425e-bdd2-66c4b594a950'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:38.687023 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-5354d415-2565-425e-bdd2-66c4b594a950 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15beef36ea194a5e889f91d4478fcf24', 'request_id': 'req-5354d415-2565-425e-bdd2-66c4b594a950'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:38.699718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-5354d415-2565-425e-bdd2-66c4b594a950 admin admin] The token cache is being invalidate because project 15beef36ea194a5e889f91d4478fcf24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:24:38.700121 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-5354d415-2565-425e-bdd2-66c4b594a950 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:38.700879 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-5354d415-2565-425e-bdd2-66c4b594a950 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:24:38.702538 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1184/2364] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:24:38 2022] DELETE /identity/v3/projects/15beef36ea194a5e889f91d4478fcf24 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:24:38.707594 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593f9a0-1070-477f-bf99-ce6945815bb8 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:38.707883 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593f9a0-1070-477f-bf99-ce6945815bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:38.708157 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593f9a0-1070-477f-bf99-ce6945815bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:38.727548 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-f593f9a0-1070-477f-bf99-ce6945815bb8 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:24:38.729300 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f593f9a0-1070-477f-bf99-ce6945815bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:24:38.807800 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1181/2365] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:24:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:24:38.816104 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf420668-7edb-456b-8c4f-8a6aa2dba75b None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:38.816407 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf420668-7edb-456b-8c4f-8a6aa2dba75b None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:38.816684 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf420668-7edb-456b-8c4f-8a6aa2dba75b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:38.841427 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-cf420668-7edb-456b-8c4f-8a6aa2dba75b None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:24:38.842720 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-cf420668-7edb-456b-8c4f-8a6aa2dba75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:24:38.851389 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1185/2366] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:24:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:24:38.856673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593f9a0-1070-477f-bf99-ce6945815bb8 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:24:38.867131 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593f9a0-1070-477f-bf99-ce6945815bb8 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:24:38.872548 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a28d8b-e28a-44d1-bb9f-a19f6ad0fb79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:24:38.873228 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a28d8b-e28a-44d1-bb9f-a19f6ad0fb79 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:38.873511 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a28d8b-e28a-44d1-bb9f-a19f6ad0fb79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:38.873785 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a28d8b-e28a-44d1-bb9f-a19f6ad0fb79 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:38.874535 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a28d8b-e28a-44d1-bb9f-a19f6ad0fb79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:24:38.874844 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a28d8b-e28a-44d1-bb9f-a19f6ad0fb79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:24:38.876528 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a28d8b-e28a-44d1-bb9f-a19f6ad0fb79 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:24:38.889560 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1182/2367] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:24:38.893756 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf420668-7edb-456b-8c4f-8a6aa2dba75b None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:24:38.903775 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf420668-7edb-456b-8c4f-8a6aa2dba75b None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:24:38.908669 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c0855c-ccc4-47db-892a-bea5f0af6cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:24:38.909348 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0855c-ccc4-47db-892a-bea5f0af6cbc admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:38.909632 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0855c-ccc4-47db-892a-bea5f0af6cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:38.909911 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0855c-ccc4-47db-892a-bea5f0af6cbc admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:38.910634 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c0855c-ccc4-47db-892a-bea5f0af6cbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:24:38.912636 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c0855c-ccc4-47db-892a-bea5f0af6cbc admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:24:38.945167 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1186/2368] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:24:38 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:24:38.946448 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a28d8b-e28a-44d1-bb9f-a19f6ad0fb79 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:24:38.958351 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a28d8b-e28a-44d1-bb9f-a19f6ad0fb79 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:24:38.990298 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:24:39.028795 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c19a5-c033-44ee-95d5-d6778904bb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:24:39.029513 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c19a5-c033-44ee-95d5-d6778904bb84 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:39.029735 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c19a5-c033-44ee-95d5-d6778904bb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:39.030004 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c19a5-c033-44ee-95d5-d6778904bb84 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:39.030699 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4c19a5-c033-44ee-95d5-d6778904bb84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:24:39.033106 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4c19a5-c033-44ee-95d5-d6778904bb84 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:24:39.056363 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1183/2369] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:24:38 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:24:39.060713 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c0855c-ccc4-47db-892a-bea5f0af6cbc admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:24:39.079674 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c0855c-ccc4-47db-892a-bea5f0af6cbc admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:24:39.088817 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0800eef-f1e5-4b2f-9220-f3979cb14e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:24:39.089993 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0800eef-f1e5-4b2f-9220-f3979cb14e32 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:39.090485 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0800eef-f1e5-4b2f-9220-f3979cb14e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:39.090986 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0800eef-f1e5-4b2f-9220-f3979cb14e32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:39.092454 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0800eef-f1e5-4b2f-9220-f3979cb14e32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:24:39.095671 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0800eef-f1e5-4b2f-9220-f3979cb14e32 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:24:39.120509 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1187/2370] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:24:39.122177 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c19a5-c033-44ee-95d5-d6778904bb84 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:24:39.139219 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c19a5-c033-44ee-95d5-d6778904bb84 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:24:39.144079 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ad5c8-b26e-483d-abe6-a7491dd1b43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:24:39.144750 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ad5c8-b26e-483d-abe6-a7491dd1b43a admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:39.145033 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ad5c8-b26e-483d-abe6-a7491dd1b43a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:39.145308 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ad5c8-b26e-483d-abe6-a7491dd1b43a admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:39.146034 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994ad5c8-b26e-483d-abe6-a7491dd1b43a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:24:39.147653 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994ad5c8-b26e-483d-abe6-a7491dd1b43a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:24:39.159060 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1184/2371] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:24:39.163042 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0800eef-f1e5-4b2f-9220-f3979cb14e32 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:24:39.174453 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0800eef-f1e5-4b2f-9220-f3979cb14e32 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:24:39.179641 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28d7d17-0794-4973-a2e7-ac801e310a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:24:39.180605 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d7d17-0794-4973-a2e7-ac801e310a6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:39.180890 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d7d17-0794-4973-a2e7-ac801e310a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:39.181303 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d7d17-0794-4973-a2e7-ac801e310a6d admin admin] PATH_INFO: `/v3/projects/b01c0d90f8d44e8c97845dbe89aa4816/users/418db90dfcef406385550d24c9c12023/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:39.199438 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28d7d17-0794-4973-a2e7-ac801e310a6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b01c0d90f8d44e8c97845dbe89aa4816, user_id=418db90dfcef406385550d24c9c12023, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:24:39.202301 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28d7d17-0794-4973-a2e7-ac801e310a6d admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:24:39.217722 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1188/2372] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:24:39 2022] PUT /identity/v3/projects/b01c0d90f8d44e8c97845dbe89aa4816/users/418db90dfcef406385550d24c9c12023/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:24:39.229945 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26616912-8d73-41a8-9da8-666d3f886d5d None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:39.230234 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26616912-8d73-41a8-9da8-666d3f886d5d None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:39.230519 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26616912-8d73-41a8-9da8-666d3f886d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:39.262776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-26616912-8d73-41a8-9da8-666d3f886d5d None None] MFA Rules not processed for user `418db90dfcef406385550d24c9c12023`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:24:39.264047 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-26616912-8d73-41a8-9da8-666d3f886d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:24:39.339743 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1185/2373] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:24:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:24:39.350878 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28d7d17-0794-4973-a2e7-ac801e310a6d admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:24:39.352682 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-f28d7d17-0794-4973-a2e7-ac801e310a6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:24:39.421205 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28d7d17-0794-4973-a2e7-ac801e310a6d admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:24:39.425058 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afcff25-955e-4577-a2d1-7db2b3a8f890 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ce9e6497fdb4068b23aa3f7b2c19536', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:24:39.425706 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afcff25-955e-4577-a2d1-7db2b3a8f890 service glance] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:39.425989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afcff25-955e-4577-a2d1-7db2b3a8f890 service glance] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:39.426268 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afcff25-955e-4577-a2d1-7db2b3a8f890 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:39.436392 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afcff25-955e-4577-a2d1-7db2b3a8f890 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:24:39.437874 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afcff25-955e-4577-a2d1-7db2b3a8f890 service glance] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:24:39.448488 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1189/2374] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:24:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:24:49.336570 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=53,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:24:49.443604 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:DELETE=6,keystone:SELECT=67,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:24:51.035466 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de815cf1-ef48-456b-9c80-fd31dc1151ce None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:24:51.036052 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de815cf1-ef48-456b-9c80-fd31dc1151ce None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:24:51.036052 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de815cf1-ef48-456b-9c80-fd31dc1151ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:24:51.056155 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-de815cf1-ef48-456b-9c80-fd31dc1151ce None None] MFA Rules not processed for user `c3871ceed47e4fcea07c5f879c8462cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:24:51.057418 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-de815cf1-ef48-456b-9c80-fd31dc1151ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:24:51.120090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1186/2375] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:24:51 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:25:01.112838 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=25 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:25:08.635060 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afcff25-955e-4577-a2d1-7db2b3a8f890 service glance] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:25:08.729487 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afcff25-955e-4577-a2d1-7db2b3a8f890 service glance] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:25:08.734166 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e812d08a-a0f0-448d-a9c7-54c65498deb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:25:08.735191 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812d08a-a0f0-448d-a9c7-54c65498deb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:08.735478 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812d08a-a0f0-448d-a9c7-54c65498deb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:08.735785 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812d08a-a0f0-448d-a9c7-54c65498deb9 admin admin] PATH_INFO: `/v3/users/418db90dfcef406385550d24c9c12023` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:08.737051 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e812d08a-a0f0-448d-a9c7-54c65498deb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=418db90dfcef406385550d24c9c12023)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:25:08.739001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e812d08a-a0f0-448d-a9c7-54c65498deb9 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:25:08.795556 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e812d08a-a0f0-448d-a9c7-54c65498deb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '418db90dfcef406385550d24c9c12023'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:25:08.796509 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e812d08a-a0f0-448d-a9c7-54c65498deb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '418db90dfcef406385550d24c9c12023'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:25:08.808447 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e812d08a-a0f0-448d-a9c7-54c65498deb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '418db90dfcef406385550d24c9c12023'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:25:08.816219 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e812d08a-a0f0-448d-a9c7-54c65498deb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '418db90dfcef406385550d24c9c12023'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:25:08.816937 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e812d08a-a0f0-448d-a9c7-54c65498deb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '418db90dfcef406385550d24c9c12023'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:25:08.830484 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1190/2376] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:25:08 2022] DELETE /identity/v3/users/418db90dfcef406385550d24c9c12023 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:25:08.835129 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fdf90e-5774-466c-98e7-608d9808d8ae None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:08.835689 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fdf90e-5774-466c-98e7-608d9808d8ae None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:08.835689 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fdf90e-5774-466c-98e7-608d9808d8ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:08.853541 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-22fdf90e-5774-466c-98e7-608d9808d8ae None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:25:08.854736 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-22fdf90e-5774-466c-98e7-608d9808d8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:25:08.914282 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1187/2377] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:25:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:25:08.925886 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e812d08a-a0f0-448d-a9c7-54c65498deb9 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:25:08.927306 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-e812d08a-a0f0-448d-a9c7-54c65498deb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:25:08.988218 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e812d08a-a0f0-448d-a9c7-54c65498deb9 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:25:08.991882 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c84460e-b749-4963-9fed-c50783a3d072 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:25:08.992527 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c84460e-b749-4963-9fed-c50783a3d072 service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:08.992808 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c84460e-b749-4963-9fed-c50783a3d072 service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:08.993078 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c84460e-b749-4963-9fed-c50783a3d072 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:09.002252 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c84460e-b749-4963-9fed-c50783a3d072 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:25:09.003739 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c84460e-b749-4963-9fed-c50783a3d072 service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:25:09.013255 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1191/2378] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:25:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:25:09.289551 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fdf90e-5774-466c-98e7-608d9808d8ae None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:25:09.291762 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-22fdf90e-5774-466c-98e7-608d9808d8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:25:09.305791 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fdf90e-5774-466c-98e7-608d9808d8ae None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:25:09.310587 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf91a957-ad46-415a-a5d9-fea74dd11c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:25:09.311463 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf91a957-ad46-415a-a5d9-fea74dd11c6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:09.311794 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf91a957-ad46-415a-a5d9-fea74dd11c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:09.312078 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf91a957-ad46-415a-a5d9-fea74dd11c6c admin admin] PATH_INFO: `/v3/projects/b01c0d90f8d44e8c97845dbe89aa4816` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:09.313271 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf91a957-ad46-415a-a5d9-fea74dd11c6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b01c0d90f8d44e8c97845dbe89aa4816)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:25:09.315249 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf91a957-ad46-415a-a5d9-fea74dd11c6c admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:25:09.367586 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-cf91a957-ad46-415a-a5d9-fea74dd11c6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b01c0d90f8d44e8c97845dbe89aa4816', 'request_id': 'req-cf91a957-ad46-415a-a5d9-fea74dd11c6c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:25:09.374118 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-cf91a957-ad46-415a-a5d9-fea74dd11c6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b01c0d90f8d44e8c97845dbe89aa4816', 'request_id': 'req-cf91a957-ad46-415a-a5d9-fea74dd11c6c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:25:09.381848 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-cf91a957-ad46-415a-a5d9-fea74dd11c6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b01c0d90f8d44e8c97845dbe89aa4816', 'request_id': 'req-cf91a957-ad46-415a-a5d9-fea74dd11c6c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:25:09.399439 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-cf91a957-ad46-415a-a5d9-fea74dd11c6c admin admin] The token cache is being invalidate because project b01c0d90f8d44e8c97845dbe89aa4816 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:25:09.399767 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-cf91a957-ad46-415a-a5d9-fea74dd11c6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:25:09.400422 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-cf91a957-ad46-415a-a5d9-fea74dd11c6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:25:09.402041 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1188/2379] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:25:09 2022] DELETE /identity/v3/projects/b01c0d90f8d44e8c97845dbe89aa4816 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:25:09.407390 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06878f74-7fd6-4324-baac-0ad7641fbfc9 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:09.407721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06878f74-7fd6-4324-baac-0ad7641fbfc9 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:09.407989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06878f74-7fd6-4324-baac-0ad7641fbfc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:09.425520 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-06878f74-7fd6-4324-baac-0ad7641fbfc9 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:25:09.426710 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-06878f74-7fd6-4324-baac-0ad7641fbfc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:25:09.444057 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:25:09.489327 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1192/2380] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:25:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:25:09.497041 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b65ba-28a0-4b8c-8d3a-8b19b4eb614b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:09.497334 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b65ba-28a0-4b8c-8d3a-8b19b4eb614b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:09.497617 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b65ba-28a0-4b8c-8d3a-8b19b4eb614b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:09.519504 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-819b65ba-28a0-4b8c-8d3a-8b19b4eb614b None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:25:09.520408 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-819b65ba-28a0-4b8c-8d3a-8b19b4eb614b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:25:09.529181 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1189/2381] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:25:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:25:09.533467 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06878f74-7fd6-4324-baac-0ad7641fbfc9 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:25:09.545024 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06878f74-7fd6-4324-baac-0ad7641fbfc9 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:25:09.550181 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3421223a-d5db-423a-ab50-3b112ee0e6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:25:09.550881 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3421223a-d5db-423a-ab50-3b112ee0e6cc admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:09.551166 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3421223a-d5db-423a-ab50-3b112ee0e6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:09.551442 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3421223a-d5db-423a-ab50-3b112ee0e6cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:09.552207 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3421223a-d5db-423a-ab50-3b112ee0e6cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:25:09.552507 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3421223a-d5db-423a-ab50-3b112ee0e6cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:25:09.554156 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3421223a-d5db-423a-ab50-3b112ee0e6cc admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:25:09.569992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1193/2382] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:25:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:25:09.571287 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b65ba-28a0-4b8c-8d3a-8b19b4eb614b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:25:09.581330 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b65ba-28a0-4b8c-8d3a-8b19b4eb614b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:25:09.586941 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec04b14-0d5b-4eee-b437-c3cc26f68a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:25:09.587851 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec04b14-0d5b-4eee-b437-c3cc26f68a4e admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:09.588129 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec04b14-0d5b-4eee-b437-c3cc26f68a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:09.588402 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec04b14-0d5b-4eee-b437-c3cc26f68a4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:09.589096 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec04b14-0d5b-4eee-b437-c3cc26f68a4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:25:09.591621 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec04b14-0d5b-4eee-b437-c3cc26f68a4e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:25:09.628591 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1190/2383] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:25:09 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:25:09.632764 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3421223a-d5db-423a-ab50-3b112ee0e6cc admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:25:09.643587 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3421223a-d5db-423a-ab50-3b112ee0e6cc admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:25:09.701609 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd3223e-30f1-4b92-bebb-d99bd5a0d455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:25:09.702270 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3223e-30f1-4b92-bebb-d99bd5a0d455 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:09.702587 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3223e-30f1-4b92-bebb-d99bd5a0d455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:09.702859 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3223e-30f1-4b92-bebb-d99bd5a0d455 admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:09.703559 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd3223e-30f1-4b92-bebb-d99bd5a0d455 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:25:09.705721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd3223e-30f1-4b92-bebb-d99bd5a0d455 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:25:09.725885 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1194/2384] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:25:09 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:25:09.729909 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec04b14-0d5b-4eee-b437-c3cc26f68a4e admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:25:09.740311 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec04b14-0d5b-4eee-b437-c3cc26f68a4e admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:25:09.744996 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8fb4c8-2621-4012-9006-9ad17c79d6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:25:09.745644 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fb4c8-2621-4012-9006-9ad17c79d6ed admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:09.745924 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fb4c8-2621-4012-9006-9ad17c79d6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:09.746691 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fb4c8-2621-4012-9006-9ad17c79d6ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:09.747415 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8fb4c8-2621-4012-9006-9ad17c79d6ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:25:09.749027 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8fb4c8-2621-4012-9006-9ad17c79d6ed admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:25:09.759015 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1191/2385] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:25:09.763365 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd3223e-30f1-4b92-bebb-d99bd5a0d455 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:25:09.774371 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd3223e-30f1-4b92-bebb-d99bd5a0d455 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:25:09.779394 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b06dd1-6135-43bc-a47c-375346f0219c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:25:09.779961 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b06dd1-6135-43bc-a47c-375346f0219c admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:09.780247 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b06dd1-6135-43bc-a47c-375346f0219c admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:09.780532 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b06dd1-6135-43bc-a47c-375346f0219c admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:09.781265 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b06dd1-6135-43bc-a47c-375346f0219c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:25:09.782878 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b06dd1-6135-43bc-a47c-375346f0219c admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:25:09.793694 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1195/2386] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:25:09.798228 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8fb4c8-2621-4012-9006-9ad17c79d6ed admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:25:09.808561 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8fb4c8-2621-4012-9006-9ad17c79d6ed admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:25:09.813782 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba69d320-f43e-437d-b694-c63bc90e882c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:25:09.814672 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba69d320-f43e-437d-b694-c63bc90e882c admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:09.814957 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba69d320-f43e-437d-b694-c63bc90e882c admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:09.815235 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba69d320-f43e-437d-b694-c63bc90e882c admin admin] PATH_INFO: `/v3/projects/7c08dd7340d94be590babe51af949697/users/7a21f5b48d554d02bcc2f8dcf4ba4cc6/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:09.831596 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba69d320-f43e-437d-b694-c63bc90e882c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c08dd7340d94be590babe51af949697, user_id=7a21f5b48d554d02bcc2f8dcf4ba4cc6, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:25:09.834238 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba69d320-f43e-437d-b694-c63bc90e882c admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:25:09.847979 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1192/2387] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:25:09 2022] PUT /identity/v3/projects/7c08dd7340d94be590babe51af949697/users/7a21f5b48d554d02bcc2f8dcf4ba4cc6/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:25:09.859262 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b33736-fad8-4aac-a923-95064d956d6a None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:09.859656 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b33736-fad8-4aac-a923-95064d956d6a None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:09.860056 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b33736-fad8-4aac-a923-95064d956d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:09.891213 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-05b33736-fad8-4aac-a923-95064d956d6a None None] MFA Rules not processed for user `7a21f5b48d554d02bcc2f8dcf4ba4cc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:25:09.892357 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-05b33736-fad8-4aac-a923-95064d956d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:25:09.966824 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1196/2388] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:25:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:25:09.977664 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba69d320-f43e-437d-b694-c63bc90e882c admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:25:09.979480 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ba69d320-f43e-437d-b694-c63bc90e882c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:25:09.994793 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:25:10.046384 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba69d320-f43e-437d-b694-c63bc90e882c admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:25:10.050201 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b75821b-9ecc-4549-867a-c5d66d62cf9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:25:10.051066 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b75821b-9ecc-4549-867a-c5d66d62cf9f service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:10.051573 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b75821b-9ecc-4549-867a-c5d66d62cf9f service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:10.052041 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b75821b-9ecc-4549-867a-c5d66d62cf9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:10.061882 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b75821b-9ecc-4549-867a-c5d66d62cf9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:25:10.063320 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b75821b-9ecc-4549-867a-c5d66d62cf9f service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:25:10.073120 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1193/2389] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:25:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:25:12.256780 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b33736-fad8-4aac-a923-95064d956d6a None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:25:12.258209 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-05b33736-fad8-4aac-a923-95064d956d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:25:12.278903 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:25:12.318677 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b33736-fad8-4aac-a923-95064d956d6a None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:25:12.322203 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc469a3d-c614-4425-9607-9d2e94ee39cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:25:12.322842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc469a3d-c614-4425-9607-9d2e94ee39cd service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:12.323119 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc469a3d-c614-4425-9607-9d2e94ee39cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:12.323391 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc469a3d-c614-4425-9607-9d2e94ee39cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:12.325712 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-fc469a3d-c614-4425-9607-9d2e94ee39cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:25:12.386232 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc469a3d-c614-4425-9607-9d2e94ee39cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:25:12.388261 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc469a3d-c614-4425-9607-9d2e94ee39cd service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:25:12.397901 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1197/2390] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:25:12 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:25:20.068346 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=21 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:25:21.718287 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b75821b-9ecc-4549-867a-c5d66d62cf9f service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:25:21.793338 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b75821b-9ecc-4549-867a-c5d66d62cf9f service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:25:21.798757 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d380e15d-99b7-43fb-a051-1f37ff27ca77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:25:21.799758 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380e15d-99b7-43fb-a051-1f37ff27ca77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:21.800087 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380e15d-99b7-43fb-a051-1f37ff27ca77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:21.800448 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d380e15d-99b7-43fb-a051-1f37ff27ca77 admin admin] PATH_INFO: `/v3/users/7a21f5b48d554d02bcc2f8dcf4ba4cc6` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:21.801736 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d380e15d-99b7-43fb-a051-1f37ff27ca77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a21f5b48d554d02bcc2f8dcf4ba4cc6)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:25:21.803911 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d380e15d-99b7-43fb-a051-1f37ff27ca77 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:25:21.856908 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d380e15d-99b7-43fb-a051-1f37ff27ca77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a21f5b48d554d02bcc2f8dcf4ba4cc6'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:25:21.857556 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d380e15d-99b7-43fb-a051-1f37ff27ca77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a21f5b48d554d02bcc2f8dcf4ba4cc6'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:25:21.869374 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d380e15d-99b7-43fb-a051-1f37ff27ca77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a21f5b48d554d02bcc2f8dcf4ba4cc6'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:25:21.876182 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d380e15d-99b7-43fb-a051-1f37ff27ca77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a21f5b48d554d02bcc2f8dcf4ba4cc6'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:25:21.884005 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d380e15d-99b7-43fb-a051-1f37ff27ca77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a21f5b48d554d02bcc2f8dcf4ba4cc6'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:25:21.890841 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1194/2391] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:25:21 2022] DELETE /identity/v3/users/7a21f5b48d554d02bcc2f8dcf4ba4cc6 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:25:21.895457 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926a7ce-084f-4b59-ae1d-c256d6887622 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:21.895971 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926a7ce-084f-4b59-ae1d-c256d6887622 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:21.896089 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4926a7ce-084f-4b59-ae1d-c256d6887622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:21.913370 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-4926a7ce-084f-4b59-ae1d-c256d6887622 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:25:21.914572 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-4926a7ce-084f-4b59-ae1d-c256d6887622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:25:21.972662 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1198/2392] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:25:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:25:21.982473 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d380e15d-99b7-43fb-a051-1f37ff27ca77 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:25:21.983943 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d380e15d-99b7-43fb-a051-1f37ff27ca77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:25:22.040179 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d380e15d-99b7-43fb-a051-1f37ff27ca77 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:25:22.043980 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103264e-daab-46a6-8515-6f8979fb18f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:25:22.044607 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103264e-daab-46a6-8515-6f8979fb18f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:22.044885 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103264e-daab-46a6-8515-6f8979fb18f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:22.045159 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103264e-daab-46a6-8515-6f8979fb18f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:22.054162 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2103264e-daab-46a6-8515-6f8979fb18f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:25:22.055664 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2103264e-daab-46a6-8515-6f8979fb18f9 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:25:22.065478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1195/2393] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:25:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:25:22.331159 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926a7ce-084f-4b59-ae1d-c256d6887622 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:25:22.332915 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-4926a7ce-084f-4b59-ae1d-c256d6887622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:25:22.345712 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4926a7ce-084f-4b59-ae1d-c256d6887622 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:25:22.350281 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c8df7e-1455-444e-bf9a-831803a84324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:25:22.351149 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c8df7e-1455-444e-bf9a-831803a84324 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:22.351429 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c8df7e-1455-444e-bf9a-831803a84324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:22.351737 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c8df7e-1455-444e-bf9a-831803a84324 admin admin] PATH_INFO: `/v3/projects/7c08dd7340d94be590babe51af949697` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:22.352958 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c8df7e-1455-444e-bf9a-831803a84324 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c08dd7340d94be590babe51af949697)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:25:22.354885 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c8df7e-1455-444e-bf9a-831803a84324 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:25:22.404184 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-88c8df7e-1455-444e-bf9a-831803a84324 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c08dd7340d94be590babe51af949697', 'request_id': 'req-88c8df7e-1455-444e-bf9a-831803a84324'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:25:22.410166 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-88c8df7e-1455-444e-bf9a-831803a84324 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c08dd7340d94be590babe51af949697', 'request_id': 'req-88c8df7e-1455-444e-bf9a-831803a84324'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:25:22.417575 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-88c8df7e-1455-444e-bf9a-831803a84324 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c08dd7340d94be590babe51af949697', 'request_id': 'req-88c8df7e-1455-444e-bf9a-831803a84324'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:25:22.419865 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:25:22.430531 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-88c8df7e-1455-444e-bf9a-831803a84324 admin admin] The token cache is being invalidate because project 7c08dd7340d94be590babe51af949697 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:25:22.430835 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-88c8df7e-1455-444e-bf9a-831803a84324 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:25:22.431643 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-88c8df7e-1455-444e-bf9a-831803a84324 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:25:22.433210 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1199/2394] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:25:22 2022] DELETE /identity/v3/projects/7c08dd7340d94be590babe51af949697 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:25:22.438544 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd48523-e81a-43ca-adb3-3c082b452782 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:22.438839 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd48523-e81a-43ca-adb3-3c082b452782 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:22.439147 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd48523-e81a-43ca-adb3-3c082b452782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:22.457202 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-7cd48523-e81a-43ca-adb3-3c082b452782 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:25:22.458476 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7cd48523-e81a-43ca-adb3-3c082b452782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:25:22.476167 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:25:22.529396 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1196/2395] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:25:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:25:22.538129 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f8b35-251f-4f38-b0bc-bf3dc74444ac None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:22.538493 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f8b35-251f-4f38-b0bc-bf3dc74444ac None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:22.538779 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f8b35-251f-4f38-b0bc-bf3dc74444ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:22.558012 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-462f8b35-251f-4f38-b0bc-bf3dc74444ac None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:25:22.559933 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-462f8b35-251f-4f38-b0bc-bf3dc74444ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:25:22.568128 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1200/2396] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:25:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:25:22.572489 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd48523-e81a-43ca-adb3-3c082b452782 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:25:22.582677 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd48523-e81a-43ca-adb3-3c082b452782 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:25:22.587256 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef9d41-1870-440d-a47a-f92860654cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:25:22.588023 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef9d41-1870-440d-a47a-f92860654cb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:22.588308 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef9d41-1870-440d-a47a-f92860654cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:22.588583 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef9d41-1870-440d-a47a-f92860654cb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:22.589314 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef9d41-1870-440d-a47a-f92860654cb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:25:22.589622 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef9d41-1870-440d-a47a-f92860654cb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:25:22.591259 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef9d41-1870-440d-a47a-f92860654cb1 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:25:22.603452 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1197/2397] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:25:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:25:22.608607 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f8b35-251f-4f38-b0bc-bf3dc74444ac None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:25:22.617998 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f8b35-251f-4f38-b0bc-bf3dc74444ac None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:25:22.623344 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c27ba4e-60f1-4ab4-997b-29da5ac26cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:25:22.624045 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c27ba4e-60f1-4ab4-997b-29da5ac26cc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:22.624349 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c27ba4e-60f1-4ab4-997b-29da5ac26cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:22.624629 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c27ba4e-60f1-4ab4-997b-29da5ac26cc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:22.625340 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c27ba4e-60f1-4ab4-997b-29da5ac26cc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:25:22.627268 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c27ba4e-60f1-4ab4-997b-29da5ac26cc1 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:25:22.657233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1201/2398] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:25:22 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:25:22.658416 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef9d41-1870-440d-a47a-f92860654cb1 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:25:22.669619 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef9d41-1870-440d-a47a-f92860654cb1 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:25:22.725615 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1686e0d-aead-4861-9e6a-103d2797f189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:25:22.726323 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1686e0d-aead-4861-9e6a-103d2797f189 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:22.726562 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1686e0d-aead-4861-9e6a-103d2797f189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:22.726840 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1686e0d-aead-4861-9e6a-103d2797f189 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:22.727560 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1686e0d-aead-4861-9e6a-103d2797f189 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:25:22.729514 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1686e0d-aead-4861-9e6a-103d2797f189 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:25:22.750933 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1198/2399] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:25:22 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:25:22.755348 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c27ba4e-60f1-4ab4-997b-29da5ac26cc1 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:25:22.766356 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c27ba4e-60f1-4ab4-997b-29da5ac26cc1 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:25:22.771410 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee5fbf3-de32-49c1-a5da-73424a617966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:25:22.772119 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee5fbf3-de32-49c1-a5da-73424a617966 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:22.772404 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee5fbf3-de32-49c1-a5da-73424a617966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:22.772681 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee5fbf3-de32-49c1-a5da-73424a617966 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:22.773422 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee5fbf3-de32-49c1-a5da-73424a617966 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:25:22.775077 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee5fbf3-de32-49c1-a5da-73424a617966 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:25:22.786607 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1202/2400] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:25:22.790743 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1686e0d-aead-4861-9e6a-103d2797f189 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:25:22.801623 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1686e0d-aead-4861-9e6a-103d2797f189 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:25:22.806563 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bf73ad-51a5-4559-87c8-adae260d8295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:25:22.807231 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf73ad-51a5-4559-87c8-adae260d8295 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:22.807513 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf73ad-51a5-4559-87c8-adae260d8295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:22.807863 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf73ad-51a5-4559-87c8-adae260d8295 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:22.809158 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bf73ad-51a5-4559-87c8-adae260d8295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:25:22.810791 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bf73ad-51a5-4559-87c8-adae260d8295 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:25:22.821576 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1199/2401] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:25:22.825890 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee5fbf3-de32-49c1-a5da-73424a617966 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:25:22.836251 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee5fbf3-de32-49c1-a5da-73424a617966 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:25:22.841344 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2087ba2-7e75-4954-a20b-948543219c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:25:22.842226 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2087ba2-7e75-4954-a20b-948543219c50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:22.842505 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2087ba2-7e75-4954-a20b-948543219c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:22.842773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2087ba2-7e75-4954-a20b-948543219c50 admin admin] PATH_INFO: `/v3/projects/b1389397378e4772a3a8270d1a0d634d/users/1b07fb7e14194d7aab3a109747b6d423/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:22.859101 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2087ba2-7e75-4954-a20b-948543219c50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1389397378e4772a3a8270d1a0d634d, user_id=1b07fb7e14194d7aab3a109747b6d423, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:25:22.861802 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2087ba2-7e75-4954-a20b-948543219c50 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:25:22.878392 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1203/2402] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:25:22 2022] PUT /identity/v3/projects/b1389397378e4772a3a8270d1a0d634d/users/1b07fb7e14194d7aab3a109747b6d423/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:25:22.889041 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e535bb-f3cd-411f-8186-7f787faeef24 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:22.889365 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e535bb-f3cd-411f-8186-7f787faeef24 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:22.889600 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e535bb-f3cd-411f-8186-7f787faeef24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:22.930518 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-60e535bb-f3cd-411f-8186-7f787faeef24 None None] MFA Rules not processed for user `1b07fb7e14194d7aab3a109747b6d423`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:25:22.932099 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-60e535bb-f3cd-411f-8186-7f787faeef24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:25:23.005299 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1200/2403] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:25:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:25:23.023389 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2087ba2-7e75-4954-a20b-948543219c50 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:25:23.025934 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-b2087ba2-7e75-4954-a20b-948543219c50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:25:23.098939 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2087ba2-7e75-4954-a20b-948543219c50 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:25:23.102876 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba7a7ed-4603-4c9b-97e9-dd64e3050c8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:25:23.103528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba7a7ed-4603-4c9b-97e9-dd64e3050c8f service cinder] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:23.103838 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba7a7ed-4603-4c9b-97e9-dd64e3050c8f service cinder] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:23.104118 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba7a7ed-4603-4c9b-97e9-dd64e3050c8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:23.113574 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba7a7ed-4603-4c9b-97e9-dd64e3050c8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:25:23.115028 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba7a7ed-4603-4c9b-97e9-dd64e3050c8f service cinder] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:25:23.124739 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1204/2404] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:25:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:25:27.317804 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e535bb-f3cd-411f-8186-7f787faeef24 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:25:27.319248 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-60e535bb-f3cd-411f-8186-7f787faeef24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:25:27.340479 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:25:27.400010 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e535bb-f3cd-411f-8186-7f787faeef24 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:25:27.405737 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55adbe40-06d5-42ad-a5f4-1e2393bad110 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:25:27.406394 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55adbe40-06d5-42ad-a5f4-1e2393bad110 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:27.406680 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55adbe40-06d5-42ad-a5f4-1e2393bad110 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:27.406961 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55adbe40-06d5-42ad-a5f4-1e2393bad110 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:27.408890 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-55adbe40-06d5-42ad-a5f4-1e2393bad110 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:25:27.492602 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55adbe40-06d5-42ad-a5f4-1e2393bad110 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:25:27.494230 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55adbe40-06d5-42ad-a5f4-1e2393bad110 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:25:27.515556 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1201/2405] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:25:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:25:31.272318 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba7a7ed-4603-4c9b-97e9-dd64e3050c8f service cinder] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:25:31.274048 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-0ba7a7ed-4603-4c9b-97e9-dd64e3050c8f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:25:31.364277 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba7a7ed-4603-4c9b-97e9-dd64e3050c8f service cinder] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:25:31.375410 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efbba41-8f6e-4a49-9ba3-6c48032f9b56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:25:31.377453 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efbba41-8f6e-4a49-9ba3-6c48032f9b56 service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:31.378015 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efbba41-8f6e-4a49-9ba3-6c48032f9b56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:31.378439 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efbba41-8f6e-4a49-9ba3-6c48032f9b56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:31.382134 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-6efbba41-8f6e-4a49-9ba3-6c48032f9b56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:25:31.402299 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efbba41-8f6e-4a49-9ba3-6c48032f9b56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:25:31.405190 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efbba41-8f6e-4a49-9ba3-6c48032f9b56 service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:25:31.427984 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1205/2406] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:25:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:25:33.665565 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13705b4-6d93-4947-97e2-c7ea56c780d2 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:33.666110 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13705b4-6d93-4947-97e2-c7ea56c780d2 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:33.666502 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13705b4-6d93-4947-97e2-c7ea56c780d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:33.721494 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-b13705b4-6d93-4947-97e2-c7ea56c780d2 None None] MFA Rules not processed for user `5e052654361f4580ab36a347610f63c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:25:33.722718 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b13705b4-6d93-4947-97e2-c7ea56c780d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:25:33.786482 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:25:33.798035 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1202/2407] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:25:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:25:33.836565 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efbba41-8f6e-4a49-9ba3-6c48032f9b56 service neutron] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:25:33.838174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-6efbba41-8f6e-4a49-9ba3-6c48032f9b56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:25:33.881728 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:25:33.928842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efbba41-8f6e-4a49-9ba3-6c48032f9b56 service neutron] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:25:33.932690 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a0b99f-a2a3-44c5-9ec9-b408caab1ace service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ce9e6497fdb4068b23aa3f7b2c19536', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:25:33.933344 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a0b99f-a2a3-44c5-9ec9-b408caab1ace service glance] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:33.933672 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a0b99f-a2a3-44c5-9ec9-b408caab1ace service glance] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:33.933946 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a0b99f-a2a3-44c5-9ec9-b408caab1ace service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:33.935856 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-98a0b99f-a2a3-44c5-9ec9-b408caab1ace service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:25:33.952367 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a0b99f-a2a3-44c5-9ec9-b408caab1ace service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:25:33.953844 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a0b99f-a2a3-44c5-9ec9-b408caab1ace service glance] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:25:33.966974 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1206/2408] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:25:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:25:42.275123 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab2033-063c-4137-a619-cff474aca32e None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:42.276144 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab2033-063c-4137-a619-cff474aca32e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:42.277082 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab2033-063c-4137-a619-cff474aca32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:42.384607 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-0fab2033-063c-4137-a619-cff474aca32e None None] MFA Rules not processed for user `a32702a67a7b49ffa153fc59251fd169`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:25:42.386180 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0fab2033-063c-4137-a619-cff474aca32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:25:42.477508 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1203/2409] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Jul 13 23:25:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:25:42.538404 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a0b99f-a2a3-44c5-9ec9-b408caab1ace service glance] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:25:42.539990 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-98a0b99f-a2a3-44c5-9ec9-b408caab1ace service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:25:42.649372 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a0b99f-a2a3-44c5-9ec9-b408caab1ace service glance] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:25:42.657980 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7d8c8e-5d83-4265-ab8a-39d780ea83ca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786fc44611804d3ca6280ffcab415569', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:25:42.658772 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d8c8e-5d83-4265-ab8a-39d780ea83ca service swift] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:42.659151 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d8c8e-5d83-4265-ab8a-39d780ea83ca service swift] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:42.660642 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d8c8e-5d83-4265-ab8a-39d780ea83ca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:42.672690 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7d8c8e-5d83-4265-ab8a-39d780ea83ca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:25:42.674798 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7d8c8e-5d83-4265-ab8a-39d780ea83ca service swift] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:25:42.686777 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1207/2410] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Jul 13 23:25:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 13 23:25:52.471599 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=46 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:25:52.681900 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=61 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:25:54.080643 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fab2033-063c-4137-a619-cff474aca32e None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:25:54.161298 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fab2033-063c-4137-a619-cff474aca32e None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:25:54.166255 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11faa3a1-92a5-4f15-8bcd-d53949048b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:25:54.167199 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11faa3a1-92a5-4f15-8bcd-d53949048b7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:54.167589 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11faa3a1-92a5-4f15-8bcd-d53949048b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:54.167931 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11faa3a1-92a5-4f15-8bcd-d53949048b7e admin admin] PATH_INFO: `/v3/users/1b07fb7e14194d7aab3a109747b6d423` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:54.170011 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11faa3a1-92a5-4f15-8bcd-d53949048b7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b07fb7e14194d7aab3a109747b6d423)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:25:54.172163 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11faa3a1-92a5-4f15-8bcd-d53949048b7e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:25:54.226330 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-11faa3a1-92a5-4f15-8bcd-d53949048b7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b07fb7e14194d7aab3a109747b6d423'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:25:54.232194 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-11faa3a1-92a5-4f15-8bcd-d53949048b7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b07fb7e14194d7aab3a109747b6d423'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:25:54.247901 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-11faa3a1-92a5-4f15-8bcd-d53949048b7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b07fb7e14194d7aab3a109747b6d423'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:25:54.259464 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-11faa3a1-92a5-4f15-8bcd-d53949048b7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b07fb7e14194d7aab3a109747b6d423'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:25:54.268288 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-11faa3a1-92a5-4f15-8bcd-d53949048b7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b07fb7e14194d7aab3a109747b6d423'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:25:54.275156 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1204/2411] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:25:54 2022] DELETE /identity/v3/users/1b07fb7e14194d7aab3a109747b6d423 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:25:54.279957 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a60132-d8ee-4a5e-8715-13625f6ddc57 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:54.280557 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a60132-d8ee-4a5e-8715-13625f6ddc57 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:54.280557 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a60132-d8ee-4a5e-8715-13625f6ddc57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:54.299676 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-40a60132-d8ee-4a5e-8715-13625f6ddc57 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:25:54.300931 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-40a60132-d8ee-4a5e-8715-13625f6ddc57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:25:54.376573 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1208/2412] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:25:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:25:54.391242 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11faa3a1-92a5-4f15-8bcd-d53949048b7e admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:25:54.392758 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-11faa3a1-92a5-4f15-8bcd-d53949048b7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:25:54.456132 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11faa3a1-92a5-4f15-8bcd-d53949048b7e admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:25:54.460438 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec505e41-75e5-45a0-b873-45914e42d1e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:25:54.461129 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec505e41-75e5-45a0-b873-45914e42d1e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:54.461464 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec505e41-75e5-45a0-b873-45914e42d1e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:54.461733 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec505e41-75e5-45a0-b873-45914e42d1e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:54.472491 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec505e41-75e5-45a0-b873-45914e42d1e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:25:54.474021 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec505e41-75e5-45a0-b873-45914e42d1e2 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:25:54.486082 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1205/2413] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:25:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:25:54.819567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a60132-d8ee-4a5e-8715-13625f6ddc57 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:25:54.821256 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-40a60132-d8ee-4a5e-8715-13625f6ddc57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:25:54.838167 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a60132-d8ee-4a5e-8715-13625f6ddc57 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:25:54.844301 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ff8e74-fa82-4ed0-a44c-49c838e47131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:25:54.851690 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff8e74-fa82-4ed0-a44c-49c838e47131 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:54.852958 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff8e74-fa82-4ed0-a44c-49c838e47131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:54.853734 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff8e74-fa82-4ed0-a44c-49c838e47131 admin admin] PATH_INFO: `/v3/projects/b1389397378e4772a3a8270d1a0d634d` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:54.856778 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ff8e74-fa82-4ed0-a44c-49c838e47131 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1389397378e4772a3a8270d1a0d634d)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:25:54.862689 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ff8e74-fa82-4ed0-a44c-49c838e47131 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:25:54.937576 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-06ff8e74-fa82-4ed0-a44c-49c838e47131 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1389397378e4772a3a8270d1a0d634d', 'request_id': 'req-06ff8e74-fa82-4ed0-a44c-49c838e47131'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:25:54.944393 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-06ff8e74-fa82-4ed0-a44c-49c838e47131 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1389397378e4772a3a8270d1a0d634d', 'request_id': 'req-06ff8e74-fa82-4ed0-a44c-49c838e47131'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:25:54.954381 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-06ff8e74-fa82-4ed0-a44c-49c838e47131 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1389397378e4772a3a8270d1a0d634d', 'request_id': 'req-06ff8e74-fa82-4ed0-a44c-49c838e47131'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:25:54.969926 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-06ff8e74-fa82-4ed0-a44c-49c838e47131 admin admin] The token cache is being invalidate because project b1389397378e4772a3a8270d1a0d634d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:25:54.970281 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-06ff8e74-fa82-4ed0-a44c-49c838e47131 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:25:54.970885 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-06ff8e74-fa82-4ed0-a44c-49c838e47131 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:25:54.972708 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1209/2414] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:25:54 2022] DELETE /identity/v3/projects/b1389397378e4772a3a8270d1a0d634d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:25:54.977579 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879fb6cf-ffe7-4410-847a-8878920afa78 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:54.977877 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879fb6cf-ffe7-4410-847a-8878920afa78 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:54.978160 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879fb6cf-ffe7-4410-847a-8878920afa78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:55.002316 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-879fb6cf-ffe7-4410-847a-8878920afa78 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:25:55.004247 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-879fb6cf-ffe7-4410-847a-8878920afa78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:25:55.054482 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:25:55.132418 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1206/2415] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:25:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:25:55.141878 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb7065-f6c1-4966-b440-44050aca9b2b None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:55.142679 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb7065-f6c1-4966-b440-44050aca9b2b None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:55.143123 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb7065-f6c1-4966-b440-44050aca9b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:55.162531 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-59fb7065-f6c1-4966-b440-44050aca9b2b None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:25:55.163790 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-59fb7065-f6c1-4966-b440-44050aca9b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:25:55.172399 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1210/2416] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:25:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:25:55.176571 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879fb6cf-ffe7-4410-847a-8878920afa78 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:25:55.187966 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879fb6cf-ffe7-4410-847a-8878920afa78 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:25:55.192710 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b88d2b0-84dc-4549-bbd4-4f5aa334d31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:25:55.193440 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88d2b0-84dc-4549-bbd4-4f5aa334d31a admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:55.193798 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88d2b0-84dc-4549-bbd4-4f5aa334d31a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:55.194170 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b88d2b0-84dc-4549-bbd4-4f5aa334d31a admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:55.194985 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b88d2b0-84dc-4549-bbd4-4f5aa334d31a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:25:55.195367 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b88d2b0-84dc-4549-bbd4-4f5aa334d31a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:25:55.197125 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b88d2b0-84dc-4549-bbd4-4f5aa334d31a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:25:55.214823 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1207/2417] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:25:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:25:55.216445 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb7065-f6c1-4966-b440-44050aca9b2b None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:25:55.227123 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb7065-f6c1-4966-b440-44050aca9b2b None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:25:55.232045 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb4a9d-0d3a-4d85-ac36-20dc0e334fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:25:55.232712 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb4a9d-0d3a-4d85-ac36-20dc0e334fc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:55.233002 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb4a9d-0d3a-4d85-ac36-20dc0e334fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:55.233287 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb4a9d-0d3a-4d85-ac36-20dc0e334fc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:55.233962 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdb4a9d-0d3a-4d85-ac36-20dc0e334fc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:25:55.236090 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdb4a9d-0d3a-4d85-ac36-20dc0e334fc5 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:25:55.270407 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1211/2418] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:25:55 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:25:55.271663 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b88d2b0-84dc-4549-bbd4-4f5aa334d31a admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:25:55.289965 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b88d2b0-84dc-4549-bbd4-4f5aa334d31a admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:25:55.372655 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923cfa64-e75f-478c-bd9a-d588d0a9996c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:25:55.373369 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923cfa64-e75f-478c-bd9a-d588d0a9996c admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:55.373652 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923cfa64-e75f-478c-bd9a-d588d0a9996c admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:55.373947 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923cfa64-e75f-478c-bd9a-d588d0a9996c admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:55.374735 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923cfa64-e75f-478c-bd9a-d588d0a9996c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:25:55.376949 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923cfa64-e75f-478c-bd9a-d588d0a9996c admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:25:55.400402 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1208/2419] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:25:55 2022] POST /identity/v3/users => generated 287 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:25:55.406250 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb4a9d-0d3a-4d85-ac36-20dc0e334fc5 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:25:55.426034 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb4a9d-0d3a-4d85-ac36-20dc0e334fc5 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:25:55.434959 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1c185e-32fa-48ea-af49-367f7578ecc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:25:55.435769 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1c185e-32fa-48ea-af49-367f7578ecc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:55.436356 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1c185e-32fa-48ea-af49-367f7578ecc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:55.436846 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1c185e-32fa-48ea-af49-367f7578ecc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:55.438217 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1c185e-32fa-48ea-af49-367f7578ecc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:25:55.440120 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1c185e-32fa-48ea-af49-367f7578ecc3 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:25:55.453211 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1212/2420] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:25:55.457694 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923cfa64-e75f-478c-bd9a-d588d0a9996c admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:25:55.471357 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923cfa64-e75f-478c-bd9a-d588d0a9996c admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:25:55.477213 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea6e80-1e6d-4ee2-a72c-5bcd765a3d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:25:55.478051 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea6e80-1e6d-4ee2-a72c-5bcd765a3d4f admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:55.478437 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea6e80-1e6d-4ee2-a72c-5bcd765a3d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:55.480390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea6e80-1e6d-4ee2-a72c-5bcd765a3d4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:55.482283 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ea6e80-1e6d-4ee2-a72c-5bcd765a3d4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:25:55.485196 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ea6e80-1e6d-4ee2-a72c-5bcd765a3d4f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:25:55.498831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1209/2421] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:25:55.503240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1c185e-32fa-48ea-af49-367f7578ecc3 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:25:55.513614 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1c185e-32fa-48ea-af49-367f7578ecc3 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:25:55.518568 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552986fd-4fca-40f3-a10d-38ba8602d255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:25:55.519465 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552986fd-4fca-40f3-a10d-38ba8602d255 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:55.519795 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552986fd-4fca-40f3-a10d-38ba8602d255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:55.520073 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552986fd-4fca-40f3-a10d-38ba8602d255 admin admin] PATH_INFO: `/v3/projects/ee4bf49f582d4479a11bdc669c44793c/users/68d7784ccf4546db938f6b7ac2f4141a/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:55.536685 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552986fd-4fca-40f3-a10d-38ba8602d255 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee4bf49f582d4479a11bdc669c44793c, user_id=68d7784ccf4546db938f6b7ac2f4141a, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:25:55.539316 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552986fd-4fca-40f3-a10d-38ba8602d255 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:25:55.554310 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1213/2422] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:25:55 2022] PUT /identity/v3/projects/ee4bf49f582d4479a11bdc669c44793c/users/68d7784ccf4546db938f6b7ac2f4141a/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:25:55.560594 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8138356f-2371-4e18-9df4-33fce0533dae None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:55.560887 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8138356f-2371-4e18-9df4-33fce0533dae None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:55.561166 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8138356f-2371-4e18-9df4-33fce0533dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:55.579211 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-8138356f-2371-4e18-9df4-33fce0533dae None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:25:55.580479 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8138356f-2371-4e18-9df4-33fce0533dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:25:55.648254 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1210/2423] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:25:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:25:55.658162 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552986fd-4fca-40f3-a10d-38ba8602d255 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:25:55.660123 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-552986fd-4fca-40f3-a10d-38ba8602d255 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:25:55.685676 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:25:55.722796 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552986fd-4fca-40f3-a10d-38ba8602d255 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:25:55.726623 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd4e1f-0175-4761-a5da-0c8a461f01e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:25:55.727282 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd4e1f-0175-4761-a5da-0c8a461f01e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:55.727590 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd4e1f-0175-4761-a5da-0c8a461f01e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:55.727875 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd4e1f-0175-4761-a5da-0c8a461f01e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:55.737958 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedd4e1f-0175-4761-a5da-0c8a461f01e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:25:55.739416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedd4e1f-0175-4761-a5da-0c8a461f01e3 service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:25:55.750077 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1214/2424] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:25:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:25:56.428425 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53c572-f1ed-47ed-9f3e-23d4034bdf71 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:56.429294 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53c572-f1ed-47ed-9f3e-23d4034bdf71 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:56.429954 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53c572-f1ed-47ed-9f3e-23d4034bdf71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:56.461892 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-bf53c572-f1ed-47ed-9f3e-23d4034bdf71 None None] MFA Rules not processed for user `68d7784ccf4546db938f6b7ac2f4141a`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:25:56.463680 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-bf53c572-f1ed-47ed-9f3e-23d4034bdf71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:25:56.486539 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:25:56.563722 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1211/2425] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:25:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:25:56.576340 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd4e1f-0175-4761-a5da-0c8a461f01e3 service neutron] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:25:56.577602 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-bedd4e1f-0175-4761-a5da-0c8a461f01e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:25:56.649058 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd4e1f-0175-4761-a5da-0c8a461f01e3 service neutron] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:25:56.653490 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140d5f9-13aa-42d3-be3b-72105330742e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:25:56.654403 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140d5f9-13aa-42d3-be3b-72105330742e service cinder] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:56.654751 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140d5f9-13aa-42d3-be3b-72105330742e service cinder] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:56.655111 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140d5f9-13aa-42d3-be3b-72105330742e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:56.664998 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9140d5f9-13aa-42d3-be3b-72105330742e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:25:56.666811 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9140d5f9-13aa-42d3-be3b-72105330742e service cinder] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:25:56.677636 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1215/2426] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:25:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:25:58.178500 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3489a54-af15-4c0b-a547-d5f4906f13d2 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:25:58.179037 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3489a54-af15-4c0b-a547-d5f4906f13d2 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:25:58.179037 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3489a54-af15-4c0b-a547-d5f4906f13d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:25:58.199394 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-e3489a54-af15-4c0b-a547-d5f4906f13d2 None None] MFA Rules not processed for user `68d7784ccf4546db938f6b7ac2f4141a`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:25:58.201198 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e3489a54-af15-4c0b-a547-d5f4906f13d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:25:58.208326 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1212/2427] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:25:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:26:00.833146 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d2873f-c296-428d-b246-4ad2aac359c8 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:26:00.834001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d2873f-c296-428d-b246-4ad2aac359c8 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:26:00.834001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d2873f-c296-428d-b246-4ad2aac359c8 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:26:00.835970 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1216/2428] 10.222.0.60 () {54 vars in 921 bytes} [Wed Jul 13 23:26:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:26:00.843322 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4fcd6-c0b5-47d2-a362-5abcb4d3c0f1 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:26:00.843941 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4fcd6-c0b5-47d2-a362-5abcb4d3c0f1 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:26:00.844085 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4fcd6-c0b5-47d2-a362-5abcb4d3c0f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:26:00.906093 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-29b4fcd6-c0b5-47d2-a362-5abcb4d3c0f1 None None] MFA Rules not processed for user `4789384d158b4995bd30e1e37e30f5f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:26:00.908203 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-29b4fcd6-c0b5-47d2-a362-5abcb4d3c0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:26:01.015813 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1213/2429] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Jul 13 23:26:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:26:01.029635 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d2873f-c296-428d-b246-4ad2aac359c8 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:26:01.031407 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-92d2873f-c296-428d-b246-4ad2aac359c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:26:01.128272 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d2873f-c296-428d-b246-4ad2aac359c8 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:26:01.134358 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100815a6-305e-4fb7-adc3-3f752a0264b7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ce9e6497fdb4068b23aa3f7b2c19536', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:26:01.134358 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100815a6-305e-4fb7-adc3-3f752a0264b7 service glance] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:26:01.134358 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100815a6-305e-4fb7-adc3-3f752a0264b7 service glance] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:26:01.134739 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100815a6-305e-4fb7-adc3-3f752a0264b7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:26:01.147299 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100815a6-305e-4fb7-adc3-3f752a0264b7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:26:01.148908 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100815a6-305e-4fb7-adc3-3f752a0264b7 service glance] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:26:01.161043 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1217/2430] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:26:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:26:01.242161 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85934872-2ee1-46f5-83c0-c26f9f57a4c6 None None] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:26:01.242454 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85934872-2ee1-46f5-83c0-c26f9f57a4c6 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:26:01.242730 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85934872-2ee1-46f5-83c0-c26f9f57a4c6 None None] PATH_INFO: `/` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:26:01.244206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1214/2431] 10.222.0.60 () {54 vars in 921 bytes} [Wed Jul 13 23:26:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:26:01.255234 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ac86b-8b17-4df4-9104-abd53d3b42aa None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:26:01.255832 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ac86b-8b17-4df4-9104-abd53d3b42aa None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:26:01.256272 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ac86b-8b17-4df4-9104-abd53d3b42aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:26:01.306211 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-f75ac86b-8b17-4df4-9104-abd53d3b42aa None None] MFA Rules not processed for user `2d670df2aff5478ebf61f55358d554de`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:26:01.308073 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-f75ac86b-8b17-4df4-9104-abd53d3b42aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:26:01.316184 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1218/2432] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Jul 13 23:26:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:26:01.327716 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85934872-2ee1-46f5-83c0-c26f9f57a4c6 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:26:01.329417 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-85934872-2ee1-46f5-83c0-c26f9f57a4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:26:01.344044 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85934872-2ee1-46f5-83c0-c26f9f57a4c6 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:26:01.347920 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5113007-479c-4da0-9005-1c5d49c7bb53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:26:01.348658 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5113007-479c-4da0-9005-1c5d49c7bb53 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:26:01.349002 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5113007-479c-4da0-9005-1c5d49c7bb53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:26:01.349340 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5113007-479c-4da0-9005-1c5d49c7bb53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:26:01.357926 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5113007-479c-4da0-9005-1c5d49c7bb53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:26:01.359457 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5113007-479c-4da0-9005-1c5d49c7bb53 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:26:01.368341 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1215/2433] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:26:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:26:01.451158 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b255742a-7804-43f0-95bb-ae77c81dd6a6 None None] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:26:01.451452 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b255742a-7804-43f0-95bb-ae77c81dd6a6 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:26:01.451785 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b255742a-7804-43f0-95bb-ae77c81dd6a6 None None] PATH_INFO: `/` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:26:01.453184 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1219/2434] 10.222.0.60 () {54 vars in 921 bytes} [Wed Jul 13 23:26:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 23:26:01.458552 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4544f-22e5-499f-8f54-0bb160495c46 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:26:01.458838 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4544f-22e5-499f-8f54-0bb160495c46 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:26:01.459112 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4544f-22e5-499f-8f54-0bb160495c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:26:01.478567 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-75d4544f-22e5-499f-8f54-0bb160495c46 None None] MFA Rules not processed for user `4789384d158b4995bd30e1e37e30f5f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:26:01.479806 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-75d4544f-22e5-499f-8f54-0bb160495c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:26:01.490334 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1216/2435] 10.222.0.60 () {58 vars in 1018 bytes} [Wed Jul 13 23:26:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:26:01.500765 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b255742a-7804-43f0-95bb-ae77c81dd6a6 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:26:01.512237 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b255742a-7804-43f0-95bb-ae77c81dd6a6 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:26:01.516082 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a469a23c-823c-4180-a9ba-a8a421015469 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:26:01.517434 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469a23c-823c-4180-a9ba-a8a421015469 service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:26:01.517434 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469a23c-823c-4180-a9ba-a8a421015469 service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:26:01.517434 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469a23c-823c-4180-a9ba-a8a421015469 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:26:01.529708 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a469a23c-823c-4180-a9ba-a8a421015469 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:26:01.531266 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a469a23c-823c-4180-a9ba-a8a421015469 service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:26:01.544853 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1220/2436] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:26:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:26:01.608026 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d4544f-22e5-499f-8f54-0bb160495c46 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:26:01.610252 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-75d4544f-22e5-499f-8f54-0bb160495c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:26:01.682848 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d4544f-22e5-499f-8f54-0bb160495c46 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:26:01.688069 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4994405-fece-43b5-a4a6-1db36a539c4f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64824ba84a024930bbc177f8a8d9a046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:26:01.688716 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4994405-fece-43b5-a4a6-1db36a539c4f service placement] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:26:01.689003 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4994405-fece-43b5-a4a6-1db36a539c4f service placement] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:26:01.689280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4994405-fece-43b5-a4a6-1db36a539c4f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:26:01.691068 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e4994405-fece-43b5-a4a6-1db36a539c4f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:26:01.702548 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4994405-fece-43b5-a4a6-1db36a539c4f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:26:01.704183 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4994405-fece-43b5-a4a6-1db36a539c4f service placement] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:26:01.709583 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:26:01.716158 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1217/2437] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:26:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:26:04.104605 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a469a23c-823c-4180-a9ba-a8a421015469 service neutron] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:26:04.114366 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a469a23c-823c-4180-a9ba-a8a421015469 service neutron] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:26:04.118074 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf42176-b07f-40b3-b908-6b00437aee79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:26:04.118721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf42176-b07f-40b3-b908-6b00437aee79 service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:26:04.118999 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf42176-b07f-40b3-b908-6b00437aee79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:26:04.119283 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf42176-b07f-40b3-b908-6b00437aee79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:26:04.122532 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-2cf42176-b07f-40b3-b908-6b00437aee79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:26:04.133503 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf42176-b07f-40b3-b908-6b00437aee79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:26:04.134951 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf42176-b07f-40b3-b908-6b00437aee79 service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:26:04.145468 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1221/2438] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:26:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:26:05.748765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887a3ab-e622-4eab-9633-1f49eeb96652 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:26:05.749252 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887a3ab-e622-4eab-9633-1f49eeb96652 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:26:05.749361 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887a3ab-e622-4eab-9633-1f49eeb96652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:26:05.766735 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-6887a3ab-e622-4eab-9633-1f49eeb96652 None None] MFA Rules not processed for user `a32702a67a7b49ffa153fc59251fd169`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:26:05.767916 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6887a3ab-e622-4eab-9633-1f49eeb96652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:26:05.823790 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1218/2439] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Jul 13 23:26:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:26:05.839905 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf42176-b07f-40b3-b908-6b00437aee79 service neutron] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:26:05.841412 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-2cf42176-b07f-40b3-b908-6b00437aee79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:26:05.867201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:26:05.912625 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf42176-b07f-40b3-b908-6b00437aee79 service neutron] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:26:05.917586 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c757199-2f90-405a-97b4-c334dfad461e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786fc44611804d3ca6280ffcab415569', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:26:05.919321 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c757199-2f90-405a-97b4-c334dfad461e service swift] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:26:05.919615 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c757199-2f90-405a-97b4-c334dfad461e service swift] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:26:05.920016 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c757199-2f90-405a-97b4-c334dfad461e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:26:05.928813 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c757199-2f90-405a-97b4-c334dfad461e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:26:05.930444 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c757199-2f90-405a-97b4-c334dfad461e service swift] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:26:05.939363 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1222/2440] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Jul 13 23:26:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 13 23:26:15.816618 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=25 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:26:15.936038 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=22 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:26:28.300292 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1467dc-1249-4d6b-9e35-03685aeb7001 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:26:28.300849 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1467dc-1249-4d6b-9e35-03685aeb7001 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:26:28.300849 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1467dc-1249-4d6b-9e35-03685aeb7001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:26:28.318770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-5d1467dc-1249-4d6b-9e35-03685aeb7001 None None] MFA Rules not processed for user `a32702a67a7b49ffa153fc59251fd169`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:26:28.320033 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5d1467dc-1249-4d6b-9e35-03685aeb7001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:26:28.328406 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1219/2441] 10.222.0.60 () {58 vars in 967 bytes} [Wed Jul 13 23:26:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:26:28.343648 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c757199-2f90-405a-97b4-c334dfad461e service swift] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:26:28.355309 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c757199-2f90-405a-97b4-c334dfad461e service swift] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:26:28.361898 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38149d6b-b512-4ad9-9b80-d358a5ca7b6f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786fc44611804d3ca6280ffcab415569', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:26:28.362905 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38149d6b-b512-4ad9-9b80-d358a5ca7b6f service swift] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:26:28.363368 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38149d6b-b512-4ad9-9b80-d358a5ca7b6f service swift] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:26:28.366188 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38149d6b-b512-4ad9-9b80-d358a5ca7b6f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:26:28.377656 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38149d6b-b512-4ad9-9b80-d358a5ca7b6f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:26:28.379622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38149d6b-b512-4ad9-9b80-d358a5ca7b6f service swift] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:26:28.391451 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1223/2442] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Jul 13 23:26:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 23:26:28.438246 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1467dc-1249-4d6b-9e35-03685aeb7001 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:26:28.439735 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5d1467dc-1249-4d6b-9e35-03685aeb7001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:26:28.456459 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1467dc-1249-4d6b-9e35-03685aeb7001 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:26:28.462575 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061c1cd-8be9-4f23-9bdd-0808388fa309 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:26:28.463234 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061c1cd-8be9-4f23-9bdd-0808388fa309 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:26:28.463549 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061c1cd-8be9-4f23-9bdd-0808388fa309 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:26:28.463825 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061c1cd-8be9-4f23-9bdd-0808388fa309 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:26:28.465839 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8061c1cd-8be9-4f23-9bdd-0808388fa309 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:26:28.480532 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061c1cd-8be9-4f23-9bdd-0808388fa309 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:26:28.482240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8061c1cd-8be9-4f23-9bdd-0808388fa309 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:26:28.493644 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1220/2443] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:26:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:26:29.930506 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ec964-3387-44d8-b2f1-3c312c3fd0e7 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:26:29.931115 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ec964-3387-44d8-b2f1-3c312c3fd0e7 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:26:29.931115 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ec964-3387-44d8-b2f1-3c312c3fd0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:26:29.950849 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-401ec964-3387-44d8-b2f1-3c312c3fd0e7 None None] MFA Rules not processed for user `68d7784ccf4546db938f6b7ac2f4141a`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:26:29.952107 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-401ec964-3387-44d8-b2f1-3c312c3fd0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:26:29.959032 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1224/2444] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:26:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:26:38.488924 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=9 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:26:39.943945 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=9 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:26:53.017396 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16bc67-691c-46b7-aa63-4743a488a276 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:26:53.017882 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16bc67-691c-46b7-aa63-4743a488a276 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:26:53.017983 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16bc67-691c-46b7-aa63-4743a488a276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:26:53.035845 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-ea16bc67-691c-46b7-aa63-4743a488a276 None None] MFA Rules not processed for user `a32702a67a7b49ffa153fc59251fd169`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:26:53.037087 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ea16bc67-691c-46b7-aa63-4743a488a276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:26:53.045287 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1221/2445] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Jul 13 23:26:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:26:53.066267 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ec964-3387-44d8-b2f1-3c312c3fd0e7 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:26:53.078173 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ec964-3387-44d8-b2f1-3c312c3fd0e7 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:26:53.083120 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea4f8b-b3df-4ec5-a1b6-603f52610948 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786fc44611804d3ca6280ffcab415569', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:26:53.083791 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea4f8b-b3df-4ec5-a1b6-603f52610948 service swift] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:26:53.084074 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea4f8b-b3df-4ec5-a1b6-603f52610948 service swift] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:26:53.084350 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea4f8b-b3df-4ec5-a1b6-603f52610948 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:26:53.093273 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daea4f8b-b3df-4ec5-a1b6-603f52610948 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:26:53.094915 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daea4f8b-b3df-4ec5-a1b6-603f52610948 service swift] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:26:53.103183 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1225/2446] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Jul 13 23:26:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 23:27:03.035988 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:27:03.099997 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=6 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:27:08.572443 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9c703-c1fc-4aba-afec-253843c97940 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:08.572913 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9c703-c1fc-4aba-afec-253843c97940 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:08.573017 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9c703-c1fc-4aba-afec-253843c97940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:08.590293 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-29c9c703-c1fc-4aba-afec-253843c97940 None None] MFA Rules not processed for user `a32702a67a7b49ffa153fc59251fd169`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:27:08.592018 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-29c9c703-c1fc-4aba-afec-253843c97940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:08.599064 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1222/2447] 10.222.0.60 () {58 vars in 967 bytes} [Wed Jul 13 23:27:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:27:08.611897 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea4f8b-b3df-4ec5-a1b6-603f52610948 service swift] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:08.622533 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea4f8b-b3df-4ec5-a1b6-603f52610948 service swift] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:08.627666 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d560d4f-d696-4b46-975c-fae956d20160 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786fc44611804d3ca6280ffcab415569', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:08.628305 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d560d4f-d696-4b46-975c-fae956d20160 service swift] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:08.628581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d560d4f-d696-4b46-975c-fae956d20160 service swift] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:08.628858 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d560d4f-d696-4b46-975c-fae956d20160 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:08.637399 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d560d4f-d696-4b46-975c-fae956d20160 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:08.638988 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d560d4f-d696-4b46-975c-fae956d20160 service swift] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:08.647039 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1226/2448] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Jul 13 23:27:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 23:27:10.615403 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c9c703-c1fc-4aba-afec-253843c97940 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:10.627595 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c9c703-c1fc-4aba-afec-253843c97940 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:10.633190 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651eeb8c-1316-49ce-8e9b-6195a631e6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:10.634072 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651eeb8c-1316-49ce-8e9b-6195a631e6c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:10.634355 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651eeb8c-1316-49ce-8e9b-6195a631e6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:10.634628 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651eeb8c-1316-49ce-8e9b-6195a631e6c5 admin admin] PATH_INFO: `/v3/users/68d7784ccf4546db938f6b7ac2f4141a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:10.635876 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651eeb8c-1316-49ce-8e9b-6195a631e6c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68d7784ccf4546db938f6b7ac2f4141a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:10.637798 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651eeb8c-1316-49ce-8e9b-6195a631e6c5 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:10.685829 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-651eeb8c-1316-49ce-8e9b-6195a631e6c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68d7784ccf4546db938f6b7ac2f4141a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:10.686495 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-651eeb8c-1316-49ce-8e9b-6195a631e6c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68d7784ccf4546db938f6b7ac2f4141a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:10.698543 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-651eeb8c-1316-49ce-8e9b-6195a631e6c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68d7784ccf4546db938f6b7ac2f4141a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:10.706489 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-651eeb8c-1316-49ce-8e9b-6195a631e6c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68d7784ccf4546db938f6b7ac2f4141a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:10.714506 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-651eeb8c-1316-49ce-8e9b-6195a631e6c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68d7784ccf4546db938f6b7ac2f4141a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:10.721276 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1223/2449] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:27:10 2022] DELETE /identity/v3/users/68d7784ccf4546db938f6b7ac2f4141a => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:27:10.898676 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d560d4f-d696-4b46-975c-fae956d20160 service swift] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:10.912153 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d560d4f-d696-4b46-975c-fae956d20160 service swift] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:10.983312 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4436751-769c-4260-9a1b-013a9a61b2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:10.984245 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4436751-769c-4260-9a1b-013a9a61b2b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:10.984531 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4436751-769c-4260-9a1b-013a9a61b2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:10.984813 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4436751-769c-4260-9a1b-013a9a61b2b7 admin admin] PATH_INFO: `/v3/projects/ee4bf49f582d4479a11bdc669c44793c` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:10.986107 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4436751-769c-4260-9a1b-013a9a61b2b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee4bf49f582d4479a11bdc669c44793c)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:10.988121 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4436751-769c-4260-9a1b-013a9a61b2b7 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:11.046239 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-f4436751-769c-4260-9a1b-013a9a61b2b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee4bf49f582d4479a11bdc669c44793c', 'request_id': 'req-f4436751-769c-4260-9a1b-013a9a61b2b7'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:11.053364 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-f4436751-769c-4260-9a1b-013a9a61b2b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee4bf49f582d4479a11bdc669c44793c', 'request_id': 'req-f4436751-769c-4260-9a1b-013a9a61b2b7'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:11.062613 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-f4436751-769c-4260-9a1b-013a9a61b2b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee4bf49f582d4479a11bdc669c44793c', 'request_id': 'req-f4436751-769c-4260-9a1b-013a9a61b2b7'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:11.074668 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-f4436751-769c-4260-9a1b-013a9a61b2b7 admin admin] The token cache is being invalidate because project ee4bf49f582d4479a11bdc669c44793c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:27:11.074971 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-f4436751-769c-4260-9a1b-013a9a61b2b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:11.075684 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-f4436751-769c-4260-9a1b-013a9a61b2b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:11.077823 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1227/2450] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:27:10 2022] DELETE /identity/v3/projects/ee4bf49f582d4479a11bdc669c44793c => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:27:11.082980 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476902c7-4976-473b-a5ac-0abf0008084c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:11.083249 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476902c7-4976-473b-a5ac-0abf0008084c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:11.083603 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476902c7-4976-473b-a5ac-0abf0008084c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:11.102613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-476902c7-4976-473b-a5ac-0abf0008084c None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:27:11.103807 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-476902c7-4976-473b-a5ac-0abf0008084c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:11.169188 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1224/2451] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:27:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:27:11.177968 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2295b20-06db-458d-be1e-30aaabcdef95 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:11.178261 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2295b20-06db-458d-be1e-30aaabcdef95 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:11.178547 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2295b20-06db-458d-be1e-30aaabcdef95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:11.198621 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-b2295b20-06db-458d-be1e-30aaabcdef95 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:27:11.199881 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-b2295b20-06db-458d-be1e-30aaabcdef95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:11.207963 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1228/2452] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:27:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:27:11.212277 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476902c7-4976-473b-a5ac-0abf0008084c None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:11.222047 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476902c7-4976-473b-a5ac-0abf0008084c None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:11.227553 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b75e08-3ead-4d86-8458-9254b56a59d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:11.228249 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b75e08-3ead-4d86-8458-9254b56a59d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:11.228533 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b75e08-3ead-4d86-8458-9254b56a59d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:11.228836 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b75e08-3ead-4d86-8458-9254b56a59d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:11.229575 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b75e08-3ead-4d86-8458-9254b56a59d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:27:11.229881 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b75e08-3ead-4d86-8458-9254b56a59d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:11.231599 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b75e08-3ead-4d86-8458-9254b56a59d8 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:11.245103 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1225/2453] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:27:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:27:11.250391 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2295b20-06db-458d-be1e-30aaabcdef95 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:11.260055 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2295b20-06db-458d-be1e-30aaabcdef95 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:11.265057 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978a004f-9c24-4c09-bb34-68f50477a919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:11.265726 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a004f-9c24-4c09-bb34-68f50477a919 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:11.266012 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a004f-9c24-4c09-bb34-68f50477a919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:11.266292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a004f-9c24-4c09-bb34-68f50477a919 admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:11.267019 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978a004f-9c24-4c09-bb34-68f50477a919 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:11.268989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978a004f-9c24-4c09-bb34-68f50477a919 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:11.299467 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1229/2454] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:27:11 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:27:11.303899 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b75e08-3ead-4d86-8458-9254b56a59d8 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:11.314923 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b75e08-3ead-4d86-8458-9254b56a59d8 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:11.365655 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:27:11.376897 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eecd22-034a-4f42-a71c-b806ce7bbea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:11.377642 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eecd22-034a-4f42-a71c-b806ce7bbea3 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:11.378099 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eecd22-034a-4f42-a71c-b806ce7bbea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:11.378379 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eecd22-034a-4f42-a71c-b806ce7bbea3 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:11.379375 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eecd22-034a-4f42-a71c-b806ce7bbea3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:11.381415 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eecd22-034a-4f42-a71c-b806ce7bbea3 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:11.405401 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1226/2455] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:27:11 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:27:11.406635 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978a004f-9c24-4c09-bb34-68f50477a919 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:11.417068 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978a004f-9c24-4c09-bb34-68f50477a919 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:11.423931 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91189842-392b-4055-b80c-3eb666bcf3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:11.424592 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91189842-392b-4055-b80c-3eb666bcf3f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:11.424871 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91189842-392b-4055-b80c-3eb666bcf3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:11.425143 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91189842-392b-4055-b80c-3eb666bcf3f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:11.425867 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91189842-392b-4055-b80c-3eb666bcf3f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:11.427475 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91189842-392b-4055-b80c-3eb666bcf3f6 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:11.438978 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1230/2456] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:27:11.443179 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eecd22-034a-4f42-a71c-b806ce7bbea3 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:11.453372 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eecd22-034a-4f42-a71c-b806ce7bbea3 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:11.458371 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bdcea1-73a8-43a5-99b7-2599afa4a184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:11.459057 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bdcea1-73a8-43a5-99b7-2599afa4a184 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:11.459374 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bdcea1-73a8-43a5-99b7-2599afa4a184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:11.459689 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bdcea1-73a8-43a5-99b7-2599afa4a184 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:11.460429 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bdcea1-73a8-43a5-99b7-2599afa4a184 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:11.462113 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bdcea1-73a8-43a5-99b7-2599afa4a184 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:11.482581 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1227/2457] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:27:11.483733 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91189842-392b-4055-b80c-3eb666bcf3f6 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:11.493980 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91189842-392b-4055-b80c-3eb666bcf3f6 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:11.499276 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ebe16b-eabf-4d63-a6e3-b204b67c74ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:11.500201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebe16b-eabf-4d63-a6e3-b204b67c74ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:11.500479 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebe16b-eabf-4d63-a6e3-b204b67c74ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:11.500752 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ebe16b-eabf-4d63-a6e3-b204b67c74ab admin admin] PATH_INFO: `/v3/projects/f96223da0efe45daaf57326c8f9d1ae1/users/0780487ed8534aa0846d6c8cd8522ada/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:11.516524 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ebe16b-eabf-4d63-a6e3-b204b67c74ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=f96223da0efe45daaf57326c8f9d1ae1, user_id=0780487ed8534aa0846d6c8cd8522ada, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:11.519607 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ebe16b-eabf-4d63-a6e3-b204b67c74ab admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:11.537701 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1231/2458] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:27:11 2022] PUT /identity/v3/projects/f96223da0efe45daaf57326c8f9d1ae1/users/0780487ed8534aa0846d6c8cd8522ada/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:27:11.543268 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc4634-427f-4cbf-b547-86f676278099 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:11.543872 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc4634-427f-4cbf-b547-86f676278099 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:11.544164 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc4634-427f-4cbf-b547-86f676278099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:11.563952 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c9bc4634-427f-4cbf-b547-86f676278099 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:27:11.565201 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c9bc4634-427f-4cbf-b547-86f676278099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:11.630767 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1228/2459] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:27:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:27:11.641250 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ebe16b-eabf-4d63-a6e3-b204b67c74ab admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:11.643003 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-95ebe16b-eabf-4d63-a6e3-b204b67c74ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:11.657904 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:27:11.706434 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ebe16b-eabf-4d63-a6e3-b204b67c74ab admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:11.710262 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb19c178-7387-407e-a4c9-b041e7f6be98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:11.710922 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19c178-7387-407e-a4c9-b041e7f6be98 service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:11.711204 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19c178-7387-407e-a4c9-b041e7f6be98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:11.711479 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19c178-7387-407e-a4c9-b041e7f6be98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:11.722449 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb19c178-7387-407e-a4c9-b041e7f6be98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:11.723947 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb19c178-7387-407e-a4c9-b041e7f6be98 service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:11.735004 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1232/2460] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:27:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:27:16.468911 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb15e2-cae7-45ed-bb42-bea3410ce4f4 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:16.469372 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb15e2-cae7-45ed-bb42-bea3410ce4f4 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:16.469372 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb15e2-cae7-45ed-bb42-bea3410ce4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:16.503662 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-fbdb15e2-cae7-45ed-bb42-bea3410ce4f4 None None] MFA Rules not processed for user `0780487ed8534aa0846d6c8cd8522ada`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:27:16.505177 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-fbdb15e2-cae7-45ed-bb42-bea3410ce4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:16.599560 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1229/2461] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:27:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:27:16.601193 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb19c178-7387-407e-a4c9-b041e7f6be98 service neutron] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:16.620167 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb19c178-7387-407e-a4c9-b041e7f6be98 service neutron] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:16.627990 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae9436d-5623-4dee-9e7a-3ceb43f70dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:16.628928 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae9436d-5623-4dee-9e7a-3ceb43f70dae admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:16.629404 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae9436d-5623-4dee-9e7a-3ceb43f70dae admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:16.630151 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae9436d-5623-4dee-9e7a-3ceb43f70dae admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:16.631133 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae9436d-5623-4dee-9e7a-3ceb43f70dae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:16.633406 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae9436d-5623-4dee-9e7a-3ceb43f70dae admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:16.690351 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb15e2-cae7-45ed-bb42-bea3410ce4f4 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:16.691761 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1233/2462] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:27:16 2022] POST /identity/v3/projects => generated 383 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:27:16.702492 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb15e2-cae7-45ed-bb42-bea3410ce4f4 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:16.795458 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50fc588-ab58-4a4c-8cf2-37d105f3b2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:16.796193 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50fc588-ab58-4a4c-8cf2-37d105f3b2ee admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:16.796470 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50fc588-ab58-4a4c-8cf2-37d105f3b2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:16.796919 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50fc588-ab58-4a4c-8cf2-37d105f3b2ee admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:16.797781 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50fc588-ab58-4a4c-8cf2-37d105f3b2ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:16.800582 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50fc588-ab58-4a4c-8cf2-37d105f3b2ee admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:16.809069 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:27:16.828665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1230/2463] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:27:16 2022] POST /identity/v3/users => generated 298 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:27:16.832763 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae9436d-5623-4dee-9e7a-3ceb43f70dae admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:16.845198 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae9436d-5623-4dee-9e7a-3ceb43f70dae admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:16.854151 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d7e54-0d14-413e-8897-3a8cac968d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:16.855367 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d7e54-0d14-413e-8897-3a8cac968d64 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:16.856133 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d7e54-0d14-413e-8897-3a8cac968d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:16.856629 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09d7e54-0d14-413e-8897-3a8cac968d64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:16.858625 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d7e54-0d14-413e-8897-3a8cac968d64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:16.861798 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09d7e54-0d14-413e-8897-3a8cac968d64 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:16.878952 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1234/2464] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:27:16 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:27:16.882997 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50fc588-ab58-4a4c-8cf2-37d105f3b2ee admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:16.894832 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50fc588-ab58-4a4c-8cf2-37d105f3b2ee admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:16.907447 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd2763d-3200-4a36-93a5-4fa3b850fcd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:16.908683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd2763d-3200-4a36-93a5-4fa3b850fcd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:16.908959 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd2763d-3200-4a36-93a5-4fa3b850fcd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:16.909235 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd2763d-3200-4a36-93a5-4fa3b850fcd6 admin admin] PATH_INFO: `/v3/projects/83937178a24e4636a96243d0fbcd67b5/users/1f2117c90756499fb687291da7a6642c/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:16.927883 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd2763d-3200-4a36-93a5-4fa3b850fcd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83937178a24e4636a96243d0fbcd67b5, user_id=1f2117c90756499fb687291da7a6642c, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:16.930663 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd2763d-3200-4a36-93a5-4fa3b850fcd6 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:16.946229 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1231/2465] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:27:16 2022] PUT /identity/v3/projects/83937178a24e4636a96243d0fbcd67b5/users/1f2117c90756499fb687291da7a6642c/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:27:16.951676 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d7e54-0d14-413e-8897-3a8cac968d64 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:16.968240 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09d7e54-0d14-413e-8897-3a8cac968d64 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:17.051511 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7911b48a-38c5-4a49-8859-adddd68d8790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:17.052224 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911b48a-38c5-4a49-8859-adddd68d8790 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:17.052506 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911b48a-38c5-4a49-8859-adddd68d8790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:17.052790 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911b48a-38c5-4a49-8859-adddd68d8790 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:17.053555 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7911b48a-38c5-4a49-8859-adddd68d8790 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:17.055204 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7911b48a-38c5-4a49-8859-adddd68d8790 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:17.066355 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1235/2466] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:27:16 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:27:17.070930 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd2763d-3200-4a36-93a5-4fa3b850fcd6 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:17.082257 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd2763d-3200-4a36-93a5-4fa3b850fcd6 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:17.088927 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ecc45-0673-48d8-bc7c-243902528da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:17.089898 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ecc45-0673-48d8-bc7c-243902528da2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:17.090184 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ecc45-0673-48d8-bc7c-243902528da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:17.090470 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ecc45-0673-48d8-bc7c-243902528da2 admin admin] PATH_INFO: `/v3/projects/83937178a24e4636a96243d0fbcd67b5/users/1f2117c90756499fb687291da7a6642c/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:17.093589 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ecc45-0673-48d8-bc7c-243902528da2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83937178a24e4636a96243d0fbcd67b5, user_id=1f2117c90756499fb687291da7a6642c, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:17.096345 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ecc45-0673-48d8-bc7c-243902528da2 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:17.110905 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1232/2467] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:27:17 2022] PUT /identity/v3/projects/83937178a24e4636a96243d0fbcd67b5/users/1f2117c90756499fb687291da7a6642c/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:27:23.895033 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3999ac9-daf6-49bf-8c13-2fc62183502c None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:23.895960 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3999ac9-daf6-49bf-8c13-2fc62183502c None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:23.895960 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3999ac9-daf6-49bf-8c13-2fc62183502c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:23.952331 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-e3999ac9-daf6-49bf-8c13-2fc62183502c None None] MFA Rules not processed for user `1f2117c90756499fb687291da7a6642c`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:27:23.953979 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-e3999ac9-daf6-49bf-8c13-2fc62183502c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:24.066886 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1236/2468] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:27:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:27:24.083651 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ecc45-0673-48d8-bc7c-243902528da2 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:24.086213 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d25ecc45-0673-48d8-bc7c-243902528da2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:24.174813 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ecc45-0673-48d8-bc7c-243902528da2 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:24.181244 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879ec29e-b2c2-48eb-a159-52e9d83a2d72 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ce9e6497fdb4068b23aa3f7b2c19536', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:24.182042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ec29e-b2c2-48eb-a159-52e9d83a2d72 service glance] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:24.182571 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ec29e-b2c2-48eb-a159-52e9d83a2d72 service glance] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:24.182975 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ec29e-b2c2-48eb-a159-52e9d83a2d72 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:24.293305 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879ec29e-b2c2-48eb-a159-52e9d83a2d72 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:24.293305 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879ec29e-b2c2-48eb-a159-52e9d83a2d72 service glance] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:24.308464 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1233/2469] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:27:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:27:24.469135 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c051aa5-d16f-4ee8-b629-270d2835c6b9 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:24.469425 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c051aa5-d16f-4ee8-b629-270d2835c6b9 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:24.469701 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c051aa5-d16f-4ee8-b629-270d2835c6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:24.479648 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:27:24.505626 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-2c051aa5-d16f-4ee8-b629-270d2835c6b9 None None] MFA Rules not processed for user `a32702a67a7b49ffa153fc59251fd169`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:27:24.507151 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-2c051aa5-d16f-4ee8-b629-270d2835c6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:24.625668 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1237/2470] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Jul 13 23:27:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:27:24.649316 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879ec29e-b2c2-48eb-a159-52e9d83a2d72 service glance] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:24.650883 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-879ec29e-b2c2-48eb-a159-52e9d83a2d72 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:24.784524 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879ec29e-b2c2-48eb-a159-52e9d83a2d72 service glance] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:24.798048 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd4452c-342a-4322-a1ca-c63cd4eba968 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786fc44611804d3ca6280ffcab415569', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:24.799020 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd4452c-342a-4322-a1ca-c63cd4eba968 service swift] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:24.799564 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd4452c-342a-4322-a1ca-c63cd4eba968 service swift] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:24.799954 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd4452c-342a-4322-a1ca-c63cd4eba968 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:24.815879 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd4452c-342a-4322-a1ca-c63cd4eba968 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:24.819744 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd4452c-342a-4322-a1ca-c63cd4eba968 service swift] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:24.824997 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:27:24.833881 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1234/2471] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Jul 13 23:27:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 23:27:28.733202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c051aa5-d16f-4ee8-b629-270d2835c6b9 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:28.735607 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-2c051aa5-d16f-4ee8-b629-270d2835c6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:28.841662 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c051aa5-d16f-4ee8-b629-270d2835c6b9 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:28.845695 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2474fa-3d10-46ba-a9fa-725c98fd628e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:28.846362 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2474fa-3d10-46ba-a9fa-725c98fd628e service cinder] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:28.846652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2474fa-3d10-46ba-a9fa-725c98fd628e service cinder] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:28.846930 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2474fa-3d10-46ba-a9fa-725c98fd628e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:28.849239 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-9a2474fa-3d10-46ba-a9fa-725c98fd628e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:28.990678 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2474fa-3d10-46ba-a9fa-725c98fd628e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:28.993902 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2474fa-3d10-46ba-a9fa-725c98fd628e service cinder] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:29.020077 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1238/2472] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:27:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:27:30.221231 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd4452c-342a-4322-a1ca-c63cd4eba968 service swift] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:30.304972 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd4452c-342a-4322-a1ca-c63cd4eba968 service swift] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:30.309199 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd53c3b-0588-4656-a3a2-ed4b43f90e67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:30.310555 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd53c3b-0588-4656-a3a2-ed4b43f90e67 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:30.310843 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd53c3b-0588-4656-a3a2-ed4b43f90e67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:30.311117 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd53c3b-0588-4656-a3a2-ed4b43f90e67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:30.313319 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6dd53c3b-0588-4656-a3a2-ed4b43f90e67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:30.414168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd53c3b-0588-4656-a3a2-ed4b43f90e67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:30.415686 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd53c3b-0588-4656-a3a2-ed4b43f90e67 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:30.430445 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1235/2473] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:27:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:27:33.886974 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2474fa-3d10-46ba-a9fa-725c98fd628e service cinder] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:33.888500 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-9a2474fa-3d10-46ba-a9fa-725c98fd628e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:33.964032 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:27:33.983489 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd53c3b-0588-4656-a3a2-ed4b43f90e67 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:33.985853 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6dd53c3b-0588-4656-a3a2-ed4b43f90e67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:34.005976 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd53c3b-0588-4656-a3a2-ed4b43f90e67 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:34.012879 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36f6580-b5f8-4b7d-8263-a2d7dd3bef73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:34.014453 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f6580-b5f8-4b7d-8263-a2d7dd3bef73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:34.014912 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f6580-b5f8-4b7d-8263-a2d7dd3bef73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:34.015512 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f6580-b5f8-4b7d-8263-a2d7dd3bef73 admin admin] PATH_INFO: `/v3/users/75652cd3bfa04c7ebae2cbea77cb5fd1` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:34.017538 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36f6580-b5f8-4b7d-8263-a2d7dd3bef73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75652cd3bfa04c7ebae2cbea77cb5fd1)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:34.019785 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36f6580-b5f8-4b7d-8263-a2d7dd3bef73 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:34.033884 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2474fa-3d10-46ba-a9fa-725c98fd628e service cinder] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:34.042974 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ffb18-f12f-460a-9138-ca5f69c97790 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64824ba84a024930bbc177f8a8d9a046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:34.044018 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ffb18-f12f-460a-9138-ca5f69c97790 service placement] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:34.044956 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ffb18-f12f-460a-9138-ca5f69c97790 service placement] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:34.050602 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ffb18-f12f-460a-9138-ca5f69c97790 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:34.050602 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-aa9ffb18-f12f-460a-9138-ca5f69c97790 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:34.060111 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9ffb18-f12f-460a-9138-ca5f69c97790 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:34.064727 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9ffb18-f12f-460a-9138-ca5f69c97790 service placement] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:34.078468 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1239/2474] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:27:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:27:34.134452 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c36f6580-b5f8-4b7d-8263-a2d7dd3bef73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75652cd3bfa04c7ebae2cbea77cb5fd1'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:34.135202 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c36f6580-b5f8-4b7d-8263-a2d7dd3bef73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75652cd3bfa04c7ebae2cbea77cb5fd1'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:34.150645 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c36f6580-b5f8-4b7d-8263-a2d7dd3bef73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75652cd3bfa04c7ebae2cbea77cb5fd1'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:34.161372 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c36f6580-b5f8-4b7d-8263-a2d7dd3bef73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75652cd3bfa04c7ebae2cbea77cb5fd1'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:34.173575 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c36f6580-b5f8-4b7d-8263-a2d7dd3bef73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75652cd3bfa04c7ebae2cbea77cb5fd1'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:34.190286 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1236/2475] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:27:33 2022] DELETE /identity/v3/users/75652cd3bfa04c7ebae2cbea77cb5fd1 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:27:34.409325 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ffb18-f12f-460a-9138-ca5f69c97790 service placement] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:34.430233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ffb18-f12f-460a-9138-ca5f69c97790 service placement] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:34.508245 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c770fb3b-c4e6-4863-a23d-82f7ec23c7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:34.509214 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c770fb3b-c4e6-4863-a23d-82f7ec23c7d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:34.509597 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c770fb3b-c4e6-4863-a23d-82f7ec23c7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:34.509944 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c770fb3b-c4e6-4863-a23d-82f7ec23c7d1 admin admin] PATH_INFO: `/v3/projects/1ce9fb9212ee408089fd63fb9e9e1de8` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:34.511252 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c770fb3b-c4e6-4863-a23d-82f7ec23c7d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ce9fb9212ee408089fd63fb9e9e1de8)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:34.515669 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c770fb3b-c4e6-4863-a23d-82f7ec23c7d1 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:34.580042 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-c770fb3b-c4e6-4863-a23d-82f7ec23c7d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ce9fb9212ee408089fd63fb9e9e1de8', 'request_id': 'req-c770fb3b-c4e6-4863-a23d-82f7ec23c7d1'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:34.587143 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-c770fb3b-c4e6-4863-a23d-82f7ec23c7d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ce9fb9212ee408089fd63fb9e9e1de8', 'request_id': 'req-c770fb3b-c4e6-4863-a23d-82f7ec23c7d1'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:34.597254 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-c770fb3b-c4e6-4863-a23d-82f7ec23c7d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ce9fb9212ee408089fd63fb9e9e1de8', 'request_id': 'req-c770fb3b-c4e6-4863-a23d-82f7ec23c7d1'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:34.612050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-c770fb3b-c4e6-4863-a23d-82f7ec23c7d1 admin admin] The token cache is being invalidate because project 1ce9fb9212ee408089fd63fb9e9e1de8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:27:34.612371 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-c770fb3b-c4e6-4863-a23d-82f7ec23c7d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:34.613145 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-c770fb3b-c4e6-4863-a23d-82f7ec23c7d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:34.614926 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1240/2476] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:27:34 2022] DELETE /identity/v3/projects/1ce9fb9212ee408089fd63fb9e9e1de8 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:27:34.620053 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf496ef-3d88-4906-b803-db5ceecbc1fe None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:34.620278 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf496ef-3d88-4906-b803-db5ceecbc1fe None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:34.620513 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf496ef-3d88-4906-b803-db5ceecbc1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:34.641108 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-eaf496ef-3d88-4906-b803-db5ceecbc1fe None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:27:34.642372 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-eaf496ef-3d88-4906-b803-db5ceecbc1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:34.657543 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:27:34.748346 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1237/2477] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:27:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:27:34.751662 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1cd8dd-8527-41b0-81f7-ea4dfa248b40 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:34.752219 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1cd8dd-8527-41b0-81f7-ea4dfa248b40 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:34.752746 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1cd8dd-8527-41b0-81f7-ea4dfa248b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:34.780440 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-ba1cd8dd-8527-41b0-81f7-ea4dfa248b40 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:27:34.781962 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-ba1cd8dd-8527-41b0-81f7-ea4dfa248b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:34.795701 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1241/2478] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:27:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:27:34.796976 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf496ef-3d88-4906-b803-db5ceecbc1fe None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:34.812976 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf496ef-3d88-4906-b803-db5ceecbc1fe None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:34.820041 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f32bad-6b28-418b-9a27-d308b4d3a875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:34.820780 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f32bad-6b28-418b-9a27-d308b4d3a875 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:34.821136 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f32bad-6b28-418b-9a27-d308b4d3a875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:34.821469 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f32bad-6b28-418b-9a27-d308b4d3a875 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:34.822273 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f32bad-6b28-418b-9a27-d308b4d3a875 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:27:34.822867 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f32bad-6b28-418b-9a27-d308b4d3a875 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:34.824873 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f32bad-6b28-418b-9a27-d308b4d3a875 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:34.842339 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1238/2479] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:27:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:27:34.844126 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1cd8dd-8527-41b0-81f7-ea4dfa248b40 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:34.858626 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1cd8dd-8527-41b0-81f7-ea4dfa248b40 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:34.873121 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cccf417-5005-4a1c-bd72-3af94bc03758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:34.873944 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccf417-5005-4a1c-bd72-3af94bc03758 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:34.875104 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccf417-5005-4a1c-bd72-3af94bc03758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:34.875629 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccf417-5005-4a1c-bd72-3af94bc03758 admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:34.877190 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cccf417-5005-4a1c-bd72-3af94bc03758 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:34.880104 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cccf417-5005-4a1c-bd72-3af94bc03758 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:34.913301 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1242/2480] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:27:34 2022] POST /identity/v3/projects => generated 357 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:27:34.914513 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f32bad-6b28-418b-9a27-d308b4d3a875 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:34.927469 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f32bad-6b28-418b-9a27-d308b4d3a875 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:34.996414 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545c480d-e058-47d7-8f21-a3e6376a04a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:34.997163 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c480d-e058-47d7-8f21-a3e6376a04a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:34.997400 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c480d-e058-47d7-8f21-a3e6376a04a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:34.997981 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c480d-e058-47d7-8f21-a3e6376a04a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:34.998383 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545c480d-e058-47d7-8f21-a3e6376a04a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:35.000344 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545c480d-e058-47d7-8f21-a3e6376a04a9 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:35.022959 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1239/2481] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:27:34 2022] POST /identity/v3/users => generated 279 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:27:35.027310 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cccf417-5005-4a1c-bd72-3af94bc03758 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:35.040843 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cccf417-5005-4a1c-bd72-3af94bc03758 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:35.047711 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85392645-7ef1-4b7c-92c9-50931d569306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:35.048390 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85392645-7ef1-4b7c-92c9-50931d569306 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:35.048673 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85392645-7ef1-4b7c-92c9-50931d569306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:35.048997 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85392645-7ef1-4b7c-92c9-50931d569306 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:35.049744 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85392645-7ef1-4b7c-92c9-50931d569306 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:35.051453 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85392645-7ef1-4b7c-92c9-50931d569306 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:35.055859 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489fca3-7e85-40c5-877c-bc66cd2598db None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:35.056149 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489fca3-7e85-40c5-877c-bc66cd2598db None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:35.056469 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489fca3-7e85-40c5-877c-bc66cd2598db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:35.064127 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1243/2482] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:27:35.068532 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85392645-7ef1-4b7c-92c9-50931d569306 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:35.079019 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85392645-7ef1-4b7c-92c9-50931d569306 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:35.079621 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-2489fca3-7e85-40c5-877c-bc66cd2598db None None] MFA Rules not processed for user `0780487ed8534aa0846d6c8cd8522ada`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:27:35.080838 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-2489fca3-7e85-40c5-877c-bc66cd2598db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:35.084287 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0334cbb-ff7d-4a9d-a6a0-2810fe7c85ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:35.085162 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0334cbb-ff7d-4a9d-a6a0-2810fe7c85ec admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:35.085595 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0334cbb-ff7d-4a9d-a6a0-2810fe7c85ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:35.085880 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0334cbb-ff7d-4a9d-a6a0-2810fe7c85ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:35.086806 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0334cbb-ff7d-4a9d-a6a0-2810fe7c85ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:35.088536 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0334cbb-ff7d-4a9d-a6a0-2810fe7c85ec admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:35.100267 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1244/2483] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:27:35.104232 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0334cbb-ff7d-4a9d-a6a0-2810fe7c85ec admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:35.111525 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:27:35.115314 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0334cbb-ff7d-4a9d-a6a0-2810fe7c85ec admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:35.121864 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b87aa4c-dad2-4dae-8e35-84be63559dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:35.122800 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87aa4c-dad2-4dae-8e35-84be63559dab admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:35.123380 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87aa4c-dad2-4dae-8e35-84be63559dab admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:35.123792 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87aa4c-dad2-4dae-8e35-84be63559dab admin admin] PATH_INFO: `/v3/projects/82ce7605bbc5498f83bbc613a0e3ba5e/users/54abe784193d42bb958657c9b2ee2d48/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:35.139880 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b87aa4c-dad2-4dae-8e35-84be63559dab admin admin] RBAC: Authorizing `identity:create_grant(project_id=82ce7605bbc5498f83bbc613a0e3ba5e, user_id=54abe784193d42bb958657c9b2ee2d48, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:35.142561 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b87aa4c-dad2-4dae-8e35-84be63559dab admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:35.144471 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1240/2484] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:27:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:27:35.160418 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1245/2485] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:27:35 2022] PUT /identity/v3/projects/82ce7605bbc5498f83bbc613a0e3ba5e/users/54abe784193d42bb958657c9b2ee2d48/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:27:35.170641 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16f65f-6c55-449c-bcb1-c3d6b4742b80 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:35.170952 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16f65f-6c55-449c-bcb1-c3d6b4742b80 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:35.171236 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16f65f-6c55-449c-bcb1-c3d6b4742b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:35.198925 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-1c16f65f-6c55-449c-bcb1-c3d6b4742b80 None None] MFA Rules not processed for user `54abe784193d42bb958657c9b2ee2d48`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:27:35.200771 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1c16f65f-6c55-449c-bcb1-c3d6b4742b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:35.235378 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:27:35.274899 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1241/2486] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:27:35 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:27:35.280177 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b87aa4c-dad2-4dae-8e35-84be63559dab admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:35.351305 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b87aa4c-dad2-4dae-8e35-84be63559dab admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:35.364513 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a639d9c7-634e-4c01-8956-4445efe94202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:35.365906 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639d9c7-634e-4c01-8956-4445efe94202 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:35.366518 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639d9c7-634e-4c01-8956-4445efe94202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:35.367039 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639d9c7-634e-4c01-8956-4445efe94202 admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:35.368429 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a639d9c7-634e-4c01-8956-4445efe94202 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:35.372251 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a639d9c7-634e-4c01-8956-4445efe94202 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:35.406648 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1246/2487] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:27:35 2022] POST /identity/v3/projects => generated 357 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:27:35.410993 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c16f65f-6c55-449c-bcb1-c3d6b4742b80 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:35.423559 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c16f65f-6c55-449c-bcb1-c3d6b4742b80 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:35.453166 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a639d9c7-634e-4c01-8956-4445efe94202 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:35.455348 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-a639d9c7-634e-4c01-8956-4445efe94202 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:35.493431 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9453b9-1555-46af-8f95-010298ef21a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:35.494106 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9453b9-1555-46af-8f95-010298ef21a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:35.494416 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9453b9-1555-46af-8f95-010298ef21a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:35.494685 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9453b9-1555-46af-8f95-010298ef21a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:35.495389 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9453b9-1555-46af-8f95-010298ef21a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:35.497536 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9453b9-1555-46af-8f95-010298ef21a5 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:35.523410 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1242/2488] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:27:35 2022] POST /identity/v3/users => generated 285 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:27:35.523986 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a639d9c7-634e-4c01-8956-4445efe94202 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:35.527600 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9453b9-1555-46af-8f95-010298ef21a5 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:35.530732 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05e6ce-6468-49bf-8e74-816e559903bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:35.531927 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05e6ce-6468-49bf-8e74-816e559903bf service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:35.532448 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05e6ce-6468-49bf-8e74-816e559903bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:35.532951 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05e6ce-6468-49bf-8e74-816e559903bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:35.536218 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-ad05e6ce-6468-49bf-8e74-816e559903bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:35.539036 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9453b9-1555-46af-8f95-010298ef21a5 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:35.544299 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76316cd-4867-4719-a631-2c759f5b41be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:35.544974 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76316cd-4867-4719-a631-2c759f5b41be admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:35.545401 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76316cd-4867-4719-a631-2c759f5b41be admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:35.545691 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76316cd-4867-4719-a631-2c759f5b41be admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:35.546436 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76316cd-4867-4719-a631-2c759f5b41be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:35.548131 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76316cd-4867-4719-a631-2c759f5b41be admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:35.554870 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad05e6ce-6468-49bf-8e74-816e559903bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:35.558457 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad05e6ce-6468-49bf-8e74-816e559903bf service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:35.560462 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1243/2489] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:27:35.564652 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76316cd-4867-4719-a631-2c759f5b41be admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:35.573402 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1247/2490] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:27:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:27:35.578765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76316cd-4867-4719-a631-2c759f5b41be admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:35.586644 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a706681-147d-40d2-ae64-454f8ab4855a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:35.588568 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a706681-147d-40d2-ae64-454f8ab4855a admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:35.589071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a706681-147d-40d2-ae64-454f8ab4855a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:35.589593 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a706681-147d-40d2-ae64-454f8ab4855a admin admin] PATH_INFO: `/v3/projects/8d4387b9510c408f8902e5937cc9ccf1/users/788b3b88b5fe4905a94d71c7cbd81388/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:35.613759 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a706681-147d-40d2-ae64-454f8ab4855a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d4387b9510c408f8902e5937cc9ccf1, user_id=788b3b88b5fe4905a94d71c7cbd81388, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:35.619137 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a706681-147d-40d2-ae64-454f8ab4855a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:35.644986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1244/2491] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:27:35 2022] PUT /identity/v3/projects/8d4387b9510c408f8902e5937cc9ccf1/users/788b3b88b5fe4905a94d71c7cbd81388/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:27:35.651230 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05e6ce-6468-49bf-8e74-816e559903bf service neutron] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:35.671946 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05e6ce-6468-49bf-8e74-816e559903bf service neutron] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:35.747983 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e01ff88-7573-465c-8436-906e4f231110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:35.749053 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01ff88-7573-465c-8436-906e4f231110 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:35.749053 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01ff88-7573-465c-8436-906e4f231110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:35.749286 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01ff88-7573-465c-8436-906e4f231110 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:35.749934 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e01ff88-7573-465c-8436-906e4f231110 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:35.752282 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e01ff88-7573-465c-8436-906e4f231110 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:35.763666 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1248/2492] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:27:35.767610 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a706681-147d-40d2-ae64-454f8ab4855a admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:35.778243 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a706681-147d-40d2-ae64-454f8ab4855a admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:35.783180 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac145f72-c879-495f-97c7-bfff0282904e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:35.784117 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac145f72-c879-495f-97c7-bfff0282904e admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:35.784396 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac145f72-c879-495f-97c7-bfff0282904e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:35.784670 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac145f72-c879-495f-97c7-bfff0282904e admin admin] PATH_INFO: `/v3/projects/8d4387b9510c408f8902e5937cc9ccf1/users/788b3b88b5fe4905a94d71c7cbd81388/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:35.786698 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac145f72-c879-495f-97c7-bfff0282904e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d4387b9510c408f8902e5937cc9ccf1, user_id=788b3b88b5fe4905a94d71c7cbd81388, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:35.789428 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac145f72-c879-495f-97c7-bfff0282904e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:35.801282 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1245/2493] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:27:35 2022] PUT /identity/v3/projects/8d4387b9510c408f8902e5937cc9ccf1/users/788b3b88b5fe4905a94d71c7cbd81388/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:27:35.813334 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9682b-35e0-4a2f-a979-f54287d9f253 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:35.813601 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9682b-35e0-4a2f-a979-f54287d9f253 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:35.813876 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f9682b-35e0-4a2f-a979-f54287d9f253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:35.820989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:27:35.846340 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-91f9682b-35e0-4a2f-a979-f54287d9f253 None None] MFA Rules not processed for user `788b3b88b5fe4905a94d71c7cbd81388`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:27:35.847649 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-91f9682b-35e0-4a2f-a979-f54287d9f253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:35.950556 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1249/2494] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:27:35 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:27:35.962222 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac145f72-c879-495f-97c7-bfff0282904e admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:35.964233 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ac145f72-c879-495f-97c7-bfff0282904e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:36.032489 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac145f72-c879-495f-97c7-bfff0282904e admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:36.036368 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef49e64-3626-430f-9843-e5beb0ef7250 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:36.037020 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef49e64-3626-430f-9843-e5beb0ef7250 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:36.037291 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef49e64-3626-430f-9843-e5beb0ef7250 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:36.037562 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef49e64-3626-430f-9843-e5beb0ef7250 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:36.048614 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef49e64-3626-430f-9843-e5beb0ef7250 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:36.050005 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef49e64-3626-430f-9843-e5beb0ef7250 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:36.061332 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1246/2495] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:27:35 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:27:36.138112 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f9682b-35e0-4a2f-a979-f54287d9f253 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:36.150267 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f9682b-35e0-4a2f-a979-f54287d9f253 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:36.154055 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82e325-3d52-4357-a854-af914f807bbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:36.154726 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82e325-3d52-4357-a854-af914f807bbb service cinder] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:36.155046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82e325-3d52-4357-a854-af914f807bbb service cinder] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:36.155306 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82e325-3d52-4357-a854-af914f807bbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:36.225600 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d82e325-3d52-4357-a854-af914f807bbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:36.227163 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d82e325-3d52-4357-a854-af914f807bbb service cinder] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:36.238666 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1250/2496] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:27:36 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:27:37.213919 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f770f9e-8086-4ead-b4b8-53bb658ecb2b None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:37.214416 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f770f9e-8086-4ead-b4b8-53bb658ecb2b None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:37.214635 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f770f9e-8086-4ead-b4b8-53bb658ecb2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:37.235203 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-6f770f9e-8086-4ead-b4b8-53bb658ecb2b None None] MFA Rules not processed for user `a32702a67a7b49ffa153fc59251fd169`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:27:37.236477 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6f770f9e-8086-4ead-b4b8-53bb658ecb2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:37.246257 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:27:37.302909 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1247/2497] 10.222.0.60 () {58 vars in 1009 bytes} [Wed Jul 13 23:27:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:27:37.316582 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82e325-3d52-4357-a854-af914f807bbb service cinder] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:37.318171 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-7d82e325-3d52-4357-a854-af914f807bbb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:37.397181 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82e325-3d52-4357-a854-af914f807bbb service cinder] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:37.402298 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c97930-9278-4246-a151-fafe068edb21 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786fc44611804d3ca6280ffcab415569', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:37.403016 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c97930-9278-4246-a151-fafe068edb21 service swift] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:37.403459 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c97930-9278-4246-a151-fafe068edb21 service swift] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:37.403774 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c97930-9278-4246-a151-fafe068edb21 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:37.412655 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c97930-9278-4246-a151-fafe068edb21 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:37.414326 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c97930-9278-4246-a151-fafe068edb21 service swift] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:37.425104 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1251/2498] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Jul 13 23:27:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 23:27:47.294699 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=18 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:27:47.420820 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=92 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:27:51.387505 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f770f9e-8086-4ead-b4b8-53bb658ecb2b None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:51.449280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f770f9e-8086-4ead-b4b8-53bb658ecb2b None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:51.454704 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0688770-216e-4f28-a74c-0c5f71fc2c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:51.455630 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0688770-216e-4f28-a74c-0c5f71fc2c98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:51.455903 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0688770-216e-4f28-a74c-0c5f71fc2c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:51.456176 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0688770-216e-4f28-a74c-0c5f71fc2c98 admin admin] PATH_INFO: `/v3/users/54abe784193d42bb958657c9b2ee2d48` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:51.457356 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0688770-216e-4f28-a74c-0c5f71fc2c98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54abe784193d42bb958657c9b2ee2d48)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:51.459279 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0688770-216e-4f28-a74c-0c5f71fc2c98 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:51.503247 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d0688770-216e-4f28-a74c-0c5f71fc2c98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54abe784193d42bb958657c9b2ee2d48'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:51.503935 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d0688770-216e-4f28-a74c-0c5f71fc2c98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54abe784193d42bb958657c9b2ee2d48'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:51.517509 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d0688770-216e-4f28-a74c-0c5f71fc2c98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54abe784193d42bb958657c9b2ee2d48'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:51.524555 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d0688770-216e-4f28-a74c-0c5f71fc2c98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54abe784193d42bb958657c9b2ee2d48'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:51.532327 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-d0688770-216e-4f28-a74c-0c5f71fc2c98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54abe784193d42bb958657c9b2ee2d48'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:51.538369 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1248/2499] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:27:51 2022] DELETE /identity/v3/users/54abe784193d42bb958657c9b2ee2d48 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:27:51.543130 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c122da9-f340-4ec0-b374-58907c57b333 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:51.543942 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c122da9-f340-4ec0-b374-58907c57b333 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:51.544052 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c122da9-f340-4ec0-b374-58907c57b333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:51.561287 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-4c122da9-f340-4ec0-b374-58907c57b333 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:27:51.562481 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-4c122da9-f340-4ec0-b374-58907c57b333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:51.639771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1252/2500] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:27:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:27:51.649807 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0688770-216e-4f28-a74c-0c5f71fc2c98 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:51.651273 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d0688770-216e-4f28-a74c-0c5f71fc2c98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:51.713517 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0688770-216e-4f28-a74c-0c5f71fc2c98 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:51.717384 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c884ae84-e5ff-4780-b7be-d9a55feff983 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:51.718031 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884ae84-e5ff-4780-b7be-d9a55feff983 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:51.718314 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884ae84-e5ff-4780-b7be-d9a55feff983 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:51.718583 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884ae84-e5ff-4780-b7be-d9a55feff983 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:51.728144 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c884ae84-e5ff-4780-b7be-d9a55feff983 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:51.729575 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c884ae84-e5ff-4780-b7be-d9a55feff983 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:51.739679 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1249/2501] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:27:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:27:52.126275 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c122da9-f340-4ec0-b374-58907c57b333 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:52.127859 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-4c122da9-f340-4ec0-b374-58907c57b333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:52.142918 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c122da9-f340-4ec0-b374-58907c57b333 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:52.148020 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625046c2-4374-4f7a-88b1-cc9e02be320a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:52.149104 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625046c2-4374-4f7a-88b1-cc9e02be320a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:52.149486 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625046c2-4374-4f7a-88b1-cc9e02be320a admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:52.149861 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625046c2-4374-4f7a-88b1-cc9e02be320a admin admin] PATH_INFO: `/v3/projects/82ce7605bbc5498f83bbc613a0e3ba5e` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:52.151194 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625046c2-4374-4f7a-88b1-cc9e02be320a admin admin] RBAC: Authorizing `identity:delete_project(project_id=82ce7605bbc5498f83bbc613a0e3ba5e)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:52.154535 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625046c2-4374-4f7a-88b1-cc9e02be320a admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:52.214184 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-625046c2-4374-4f7a-88b1-cc9e02be320a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82ce7605bbc5498f83bbc613a0e3ba5e', 'request_id': 'req-625046c2-4374-4f7a-88b1-cc9e02be320a'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:52.233700 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-625046c2-4374-4f7a-88b1-cc9e02be320a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82ce7605bbc5498f83bbc613a0e3ba5e', 'request_id': 'req-625046c2-4374-4f7a-88b1-cc9e02be320a'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:52.248218 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-625046c2-4374-4f7a-88b1-cc9e02be320a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82ce7605bbc5498f83bbc613a0e3ba5e', 'request_id': 'req-625046c2-4374-4f7a-88b1-cc9e02be320a'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:52.265590 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-625046c2-4374-4f7a-88b1-cc9e02be320a admin admin] The token cache is being invalidate because project 82ce7605bbc5498f83bbc613a0e3ba5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:27:52.265991 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-625046c2-4374-4f7a-88b1-cc9e02be320a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:52.266976 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-625046c2-4374-4f7a-88b1-cc9e02be320a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:52.271586 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1253/2502] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:27:52 2022] DELETE /identity/v3/projects/82ce7605bbc5498f83bbc613a0e3ba5e => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:27:52.273101 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c884ae84-e5ff-4780-b7be-d9a55feff983 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:52.284313 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c884ae84-e5ff-4780-b7be-d9a55feff983 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:52.305842 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:27:52.370915 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05437e0e-08a3-46b0-972c-50ae3830ef23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:52.371093 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05437e0e-08a3-46b0-972c-50ae3830ef23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:52.371365 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05437e0e-08a3-46b0-972c-50ae3830ef23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:52.371671 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05437e0e-08a3-46b0-972c-50ae3830ef23 admin admin] PATH_INFO: `/v3/users/788b3b88b5fe4905a94d71c7cbd81388` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:52.373733 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05437e0e-08a3-46b0-972c-50ae3830ef23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=788b3b88b5fe4905a94d71c7cbd81388)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:52.375623 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05437e0e-08a3-46b0-972c-50ae3830ef23 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:52.438529 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-05437e0e-08a3-46b0-972c-50ae3830ef23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '788b3b88b5fe4905a94d71c7cbd81388'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:52.440343 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-05437e0e-08a3-46b0-972c-50ae3830ef23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '788b3b88b5fe4905a94d71c7cbd81388'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:52.453883 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-05437e0e-08a3-46b0-972c-50ae3830ef23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '788b3b88b5fe4905a94d71c7cbd81388'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:52.461182 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-05437e0e-08a3-46b0-972c-50ae3830ef23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '788b3b88b5fe4905a94d71c7cbd81388'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:52.469390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-05437e0e-08a3-46b0-972c-50ae3830ef23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '788b3b88b5fe4905a94d71c7cbd81388'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:52.476173 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1250/2503] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:27:52 2022] DELETE /identity/v3/users/788b3b88b5fe4905a94d71c7cbd81388 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:27:52.734211 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625046c2-4374-4f7a-88b1-cc9e02be320a admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:52.735756 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-625046c2-4374-4f7a-88b1-cc9e02be320a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:52.796736 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625046c2-4374-4f7a-88b1-cc9e02be320a admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:52.840451 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:27:52.854388 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b620af80-8efe-498e-aff5-e2f8655ad30a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:52.854661 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b620af80-8efe-498e-aff5-e2f8655ad30a service cinder] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:52.854832 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b620af80-8efe-498e-aff5-e2f8655ad30a service cinder] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:52.855141 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b620af80-8efe-498e-aff5-e2f8655ad30a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:52.858102 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-b620af80-8efe-498e-aff5-e2f8655ad30a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:52.888476 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05437e0e-08a3-46b0-972c-50ae3830ef23 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:52.890176 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-05437e0e-08a3-46b0-972c-50ae3830ef23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:52.930502 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b620af80-8efe-498e-aff5-e2f8655ad30a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:52.931911 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b620af80-8efe-498e-aff5-e2f8655ad30a service cinder] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:52.942018 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1254/2504] 10.222.0.60 () {60 vars in 1396 bytes} [Wed Jul 13 23:27:52 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3250 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:27:52.975494 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05437e0e-08a3-46b0-972c-50ae3830ef23 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:52.981424 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51982ba8-5703-4b63-a778-c2c64d44c938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:52.982349 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51982ba8-5703-4b63-a778-c2c64d44c938 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:52.982634 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51982ba8-5703-4b63-a778-c2c64d44c938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:52.982965 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51982ba8-5703-4b63-a778-c2c64d44c938 admin admin] PATH_INFO: `/v3/projects/8d4387b9510c408f8902e5937cc9ccf1` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:52.984228 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51982ba8-5703-4b63-a778-c2c64d44c938 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d4387b9510c408f8902e5937cc9ccf1)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:52.986242 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51982ba8-5703-4b63-a778-c2c64d44c938 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:53.037325 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:27:53.041670 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-51982ba8-5703-4b63-a778-c2c64d44c938 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d4387b9510c408f8902e5937cc9ccf1', 'request_id': 'req-51982ba8-5703-4b63-a778-c2c64d44c938'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:53.054966 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-51982ba8-5703-4b63-a778-c2c64d44c938 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d4387b9510c408f8902e5937cc9ccf1', 'request_id': 'req-51982ba8-5703-4b63-a778-c2c64d44c938'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:53.070607 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-51982ba8-5703-4b63-a778-c2c64d44c938 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d4387b9510c408f8902e5937cc9ccf1', 'request_id': 'req-51982ba8-5703-4b63-a778-c2c64d44c938'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:53.084475 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-51982ba8-5703-4b63-a778-c2c64d44c938 admin admin] The token cache is being invalidate because project 8d4387b9510c408f8902e5937cc9ccf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:27:53.084914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-51982ba8-5703-4b63-a778-c2c64d44c938 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:53.085669 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-51982ba8-5703-4b63-a778-c2c64d44c938 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:53.091151 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1251/2505] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:27:52 2022] DELETE /identity/v3/projects/8d4387b9510c408f8902e5937cc9ccf1 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:27:53.092505 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad88f0-81a1-4cbf-9dc6-5cdedeb29e95 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:53.092793 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad88f0-81a1-4cbf-9dc6-5cdedeb29e95 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:53.093085 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad88f0-81a1-4cbf-9dc6-5cdedeb29e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:53.113609 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-94ad88f0-81a1-4cbf-9dc6-5cdedeb29e95 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:27:53.114286 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-94ad88f0-81a1-4cbf-9dc6-5cdedeb29e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:53.219191 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1255/2506] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:27:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:27:53.227051 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d4c9b-d7bd-410f-99b9-8ebd00b4ac0c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:53.227360 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d4c9b-d7bd-410f-99b9-8ebd00b4ac0c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:53.227694 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d4c9b-d7bd-410f-99b9-8ebd00b4ac0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:53.249843 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c59d4c9b-d7bd-410f-99b9-8ebd00b4ac0c None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:27:53.251510 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c59d4c9b-d7bd-410f-99b9-8ebd00b4ac0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:53.260022 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1252/2507] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:27:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:27:53.264229 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad88f0-81a1-4cbf-9dc6-5cdedeb29e95 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:53.274376 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad88f0-81a1-4cbf-9dc6-5cdedeb29e95 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:53.279325 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c3eca8-b8fc-455d-ac8e-e8e6e784da34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:53.280037 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3eca8-b8fc-455d-ac8e-e8e6e784da34 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:53.280323 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3eca8-b8fc-455d-ac8e-e8e6e784da34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:53.280603 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3eca8-b8fc-455d-ac8e-e8e6e784da34 admin admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:53.281344 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c3eca8-b8fc-455d-ac8e-e8e6e784da34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:27:53.281651 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c3eca8-b8fc-455d-ac8e-e8e6e784da34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:53.283486 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c3eca8-b8fc-455d-ac8e-e8e6e784da34 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:53.298915 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1256/2508] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:27:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:27:53.303136 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d4c9b-d7bd-410f-99b9-8ebd00b4ac0c None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:53.313538 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d4c9b-d7bd-410f-99b9-8ebd00b4ac0c None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:53.318342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a696d4-5417-4a78-9ee5-2ba330326680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:53.319009 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a696d4-5417-4a78-9ee5-2ba330326680 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:53.319291 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a696d4-5417-4a78-9ee5-2ba330326680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:53.319595 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a696d4-5417-4a78-9ee5-2ba330326680 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:53.320315 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a696d4-5417-4a78-9ee5-2ba330326680 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:53.322339 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a696d4-5417-4a78-9ee5-2ba330326680 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:53.357062 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1253/2509] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:27:53 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:27:53.361856 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c3eca8-b8fc-455d-ac8e-e8e6e784da34 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:53.376752 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c3eca8-b8fc-455d-ac8e-e8e6e784da34 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:53.444581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efff414-8071-4211-be7d-d31a2d9ccc05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:53.445321 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efff414-8071-4211-be7d-d31a2d9ccc05 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:53.445678 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efff414-8071-4211-be7d-d31a2d9ccc05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:53.445980 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efff414-8071-4211-be7d-d31a2d9ccc05 admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:53.446743 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efff414-8071-4211-be7d-d31a2d9ccc05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:53.448796 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efff414-8071-4211-be7d-d31a2d9ccc05 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:53.470727 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1257/2510] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:27:53 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:27:53.474569 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a696d4-5417-4a78-9ee5-2ba330326680 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:53.489242 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a696d4-5417-4a78-9ee5-2ba330326680 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:53.494383 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33359195-4633-4db0-8648-aa4a5559fd51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:53.495118 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33359195-4633-4db0-8648-aa4a5559fd51 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:53.495486 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33359195-4633-4db0-8648-aa4a5559fd51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:53.495860 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33359195-4633-4db0-8648-aa4a5559fd51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:53.496658 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33359195-4633-4db0-8648-aa4a5559fd51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:53.498893 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33359195-4633-4db0-8648-aa4a5559fd51 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:53.510202 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1254/2511] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:27:53.514523 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efff414-8071-4211-be7d-d31a2d9ccc05 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:53.528901 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efff414-8071-4211-be7d-d31a2d9ccc05 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:53.535279 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f4ded-ed06-4d3f-8cb1-b52927fd61a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:53.536415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f4ded-ed06-4d3f-8cb1-b52927fd61a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:53.536992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f4ded-ed06-4d3f-8cb1-b52927fd61a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:53.537475 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f4ded-ed06-4d3f-8cb1-b52927fd61a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:53.538522 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f4ded-ed06-4d3f-8cb1-b52927fd61a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:53.540581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f4ded-ed06-4d3f-8cb1-b52927fd61a4 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:53.548792 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:27:53.560076 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1258/2512] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:27:53.564204 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33359195-4633-4db0-8648-aa4a5559fd51 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:53.576122 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33359195-4633-4db0-8648-aa4a5559fd51 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:53.581141 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8ad4ce-21d6-4ff1-92c2-519109adbe5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:53.582194 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ad4ce-21d6-4ff1-92c2-519109adbe5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:53.582468 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ad4ce-21d6-4ff1-92c2-519109adbe5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:53.582735 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ad4ce-21d6-4ff1-92c2-519109adbe5b admin admin] PATH_INFO: `/v3/projects/fedca4c4f14f4ed1a4492caa770a871f/users/83353f8e0fb64bab8528605ec7f72d8a/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:53.600929 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8ad4ce-21d6-4ff1-92c2-519109adbe5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fedca4c4f14f4ed1a4492caa770a871f, user_id=83353f8e0fb64bab8528605ec7f72d8a, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:53.604292 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8ad4ce-21d6-4ff1-92c2-519109adbe5b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:53.624638 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1255/2513] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:27:53 2022] PUT /identity/v3/projects/fedca4c4f14f4ed1a4492caa770a871f/users/83353f8e0fb64bab8528605ec7f72d8a/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:27:53.636548 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e30e0-7103-4b1e-bac1-e757975e211d None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:53.636820 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e30e0-7103-4b1e-bac1-e757975e211d None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:53.637083 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e30e0-7103-4b1e-bac1-e757975e211d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:53.679854 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-d50e30e0-7103-4b1e-bac1-e757975e211d None None] MFA Rules not processed for user `83353f8e0fb64bab8528605ec7f72d8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:27:53.681082 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-d50e30e0-7103-4b1e-bac1-e757975e211d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:53.753566 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1259/2514] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:27:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:27:53.758247 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8ad4ce-21d6-4ff1-92c2-519109adbe5b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:53.821795 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8ad4ce-21d6-4ff1-92c2-519109adbe5b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:53.826424 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886cedd6-519f-488c-9f9f-bfc3a54142b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:53.827075 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886cedd6-519f-488c-9f9f-bfc3a54142b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:53.827354 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886cedd6-519f-488c-9f9f-bfc3a54142b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:53.827662 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886cedd6-519f-488c-9f9f-bfc3a54142b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:53.828392 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886cedd6-519f-488c-9f9f-bfc3a54142b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:53.830314 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886cedd6-519f-488c-9f9f-bfc3a54142b7 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:53.861980 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1256/2515] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:27:53 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:27:53.863026 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e30e0-7103-4b1e-bac1-e757975e211d None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:53.874087 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e30e0-7103-4b1e-bac1-e757975e211d None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:53.934430 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf42651-37b3-4bbb-bbd4-17bd5304a5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:53.935102 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf42651-37b3-4bbb-bbd4-17bd5304a5f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:53.935379 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf42651-37b3-4bbb-bbd4-17bd5304a5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:53.935694 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf42651-37b3-4bbb-bbd4-17bd5304a5f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:53.936391 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf42651-37b3-4bbb-bbd4-17bd5304a5f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:53.938340 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf42651-37b3-4bbb-bbd4-17bd5304a5f7 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:53.963150 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1260/2516] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:27:53 2022] POST /identity/v3/users => generated 299 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:27:53.969070 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886cedd6-519f-488c-9f9f-bfc3a54142b7 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:53.988463 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886cedd6-519f-488c-9f9f-bfc3a54142b7 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:53.997512 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e34979-daf1-4415-91b7-f02557ee9267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:53.998673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e34979-daf1-4415-91b7-f02557ee9267 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:53.999232 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e34979-daf1-4415-91b7-f02557ee9267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:53.999805 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e34979-daf1-4415-91b7-f02557ee9267 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:54.001101 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e34979-daf1-4415-91b7-f02557ee9267 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:54.004370 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e34979-daf1-4415-91b7-f02557ee9267 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:54.028043 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1257/2517] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:27:54.033056 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf42651-37b3-4bbb-bbd4-17bd5304a5f7 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:54.044530 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf42651-37b3-4bbb-bbd4-17bd5304a5f7 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:54.049459 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18596f9a-8482-4507-98a9-8c2b318a50ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:54.050533 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18596f9a-8482-4507-98a9-8c2b318a50ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:54.050953 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18596f9a-8482-4507-98a9-8c2b318a50ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:54.051233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18596f9a-8482-4507-98a9-8c2b318a50ea admin admin] PATH_INFO: `/v3/projects/e30a7e8182154a6e88d588c9dc277f44/users/f7531d5a1a1141a08aad242ddd2fe802/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:54.067618 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18596f9a-8482-4507-98a9-8c2b318a50ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=e30a7e8182154a6e88d588c9dc277f44, user_id=f7531d5a1a1141a08aad242ddd2fe802, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:54.070324 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18596f9a-8482-4507-98a9-8c2b318a50ea admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:54.085309 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1261/2518] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:27:54 2022] PUT /identity/v3/projects/e30a7e8182154a6e88d588c9dc277f44/users/f7531d5a1a1141a08aad242ddd2fe802/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:27:54.089451 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e34979-daf1-4415-91b7-f02557ee9267 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:54.100705 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e34979-daf1-4415-91b7-f02557ee9267 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:54.153019 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:27:54.177454 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8e3278-d3b3-43ed-8912-f618179ec876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:54.178376 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e3278-d3b3-43ed-8912-f618179ec876 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:54.179131 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e3278-d3b3-43ed-8912-f618179ec876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:54.179644 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e3278-d3b3-43ed-8912-f618179ec876 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:54.180537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8e3278-d3b3-43ed-8912-f618179ec876 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:54.182390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8e3278-d3b3-43ed-8912-f618179ec876 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:54.197302 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1258/2519] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:27:54.199003 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18596f9a-8482-4507-98a9-8c2b318a50ea admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:54.219269 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18596f9a-8482-4507-98a9-8c2b318a50ea admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:54.219269 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c8a843-9745-467d-8d8e-06e10e172166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:54.219269 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8a843-9745-467d-8d8e-06e10e172166 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:54.219269 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8a843-9745-467d-8d8e-06e10e172166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:54.219747 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c8a843-9745-467d-8d8e-06e10e172166 admin admin] PATH_INFO: `/v3/projects/e30a7e8182154a6e88d588c9dc277f44/users/f7531d5a1a1141a08aad242ddd2fe802/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:54.220864 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c8a843-9745-467d-8d8e-06e10e172166 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e30a7e8182154a6e88d588c9dc277f44, user_id=f7531d5a1a1141a08aad242ddd2fe802, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:54.223521 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c8a843-9745-467d-8d8e-06e10e172166 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:54.237468 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1262/2520] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:27:54 2022] PUT /identity/v3/projects/e30a7e8182154a6e88d588c9dc277f44/users/f7531d5a1a1141a08aad242ddd2fe802/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:27:54.248512 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3192581-aa6d-4695-917b-e9358695f640 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:54.249139 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3192581-aa6d-4695-917b-e9358695f640 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:54.249871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3192581-aa6d-4695-917b-e9358695f640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:54.277069 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c3192581-aa6d-4695-917b-e9358695f640 None None] MFA Rules not processed for user `f7531d5a1a1141a08aad242ddd2fe802`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:27:54.278887 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c3192581-aa6d-4695-917b-e9358695f640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:54.374188 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1259/2521] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:27:54 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:27:54.385569 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c8a843-9745-467d-8d8e-06e10e172166 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:54.388363 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-d6c8a843-9745-467d-8d8e-06e10e172166 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:54.439608 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:27:54.453690 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c8a843-9745-467d-8d8e-06e10e172166 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:54.457375 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92e477-8474-4541-89d7-1e82797534bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:54.458025 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92e477-8474-4541-89d7-1e82797534bd service cinder] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:54.458307 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92e477-8474-4541-89d7-1e82797534bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:54.458580 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92e477-8474-4541-89d7-1e82797534bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:54.468037 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a92e477-8474-4541-89d7-1e82797534bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:54.469540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a92e477-8474-4541-89d7-1e82797534bd service cinder] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:54.479133 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1263/2522] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:27:54 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:27:54.509176 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3192581-aa6d-4695-917b-e9358695f640 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:54.590207 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3192581-aa6d-4695-917b-e9358695f640 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:54.595871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5525d39-35c2-4a4c-b43a-ee7c3e242c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:54.596917 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5525d39-35c2-4a4c-b43a-ee7c3e242c55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:54.597402 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5525d39-35c2-4a4c-b43a-ee7c3e242c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:54.597865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5525d39-35c2-4a4c-b43a-ee7c3e242c55 admin admin] PATH_INFO: `/v3/users/83353f8e0fb64bab8528605ec7f72d8a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:54.599277 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5525d39-35c2-4a4c-b43a-ee7c3e242c55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83353f8e0fb64bab8528605ec7f72d8a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:54.601476 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5525d39-35c2-4a4c-b43a-ee7c3e242c55 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:54.659991 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a5525d39-35c2-4a4c-b43a-ee7c3e242c55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83353f8e0fb64bab8528605ec7f72d8a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:54.660817 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a5525d39-35c2-4a4c-b43a-ee7c3e242c55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83353f8e0fb64bab8528605ec7f72d8a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:54.675418 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a5525d39-35c2-4a4c-b43a-ee7c3e242c55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83353f8e0fb64bab8528605ec7f72d8a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:54.677013 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:27:54.697120 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a5525d39-35c2-4a4c-b43a-ee7c3e242c55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83353f8e0fb64bab8528605ec7f72d8a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:54.707026 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-a5525d39-35c2-4a4c-b43a-ee7c3e242c55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83353f8e0fb64bab8528605ec7f72d8a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:54.718680 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1260/2523] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:27:54 2022] DELETE /identity/v3/users/83353f8e0fb64bab8528605ec7f72d8a => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:27:54.720046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23713bd9-fa50-4df3-bbef-d21a7ff3194f None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:54.720290 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23713bd9-fa50-4df3-bbef-d21a7ff3194f None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:54.720576 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23713bd9-fa50-4df3-bbef-d21a7ff3194f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:54.743292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-23713bd9-fa50-4df3-bbef-d21a7ff3194f None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:27:54.744696 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-23713bd9-fa50-4df3-bbef-d21a7ff3194f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:54.842076 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1264/2524] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:27:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:27:54.851912 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5525d39-35c2-4a4c-b43a-ee7c3e242c55 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:54.854345 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a5525d39-35c2-4a4c-b43a-ee7c3e242c55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:54.930321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5525d39-35c2-4a4c-b43a-ee7c3e242c55 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:54.933936 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25efe381-40f7-4359-bf45-f9fbe81fed38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:54.935146 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25efe381-40f7-4359-bf45-f9fbe81fed38 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:54.935431 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25efe381-40f7-4359-bf45-f9fbe81fed38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:54.935740 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25efe381-40f7-4359-bf45-f9fbe81fed38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:54.945509 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25efe381-40f7-4359-bf45-f9fbe81fed38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:54.947148 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25efe381-40f7-4359-bf45-f9fbe81fed38 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:54.957179 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1261/2525] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:27:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:27:55.250132 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23713bd9-fa50-4df3-bbef-d21a7ff3194f None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:55.251662 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-23713bd9-fa50-4df3-bbef-d21a7ff3194f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:55.267981 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23713bd9-fa50-4df3-bbef-d21a7ff3194f None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:55.274552 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba88fa-5006-4bb3-bac3-a4515fcf5938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:55.275466 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba88fa-5006-4bb3-bac3-a4515fcf5938 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:55.275801 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba88fa-5006-4bb3-bac3-a4515fcf5938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:55.276225 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba88fa-5006-4bb3-bac3-a4515fcf5938 admin admin] PATH_INFO: `/v3/projects/fedca4c4f14f4ed1a4492caa770a871f` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:55.277784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caba88fa-5006-4bb3-bac3-a4515fcf5938 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fedca4c4f14f4ed1a4492caa770a871f)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:55.280800 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caba88fa-5006-4bb3-bac3-a4515fcf5938 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:55.336208 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-caba88fa-5006-4bb3-bac3-a4515fcf5938 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fedca4c4f14f4ed1a4492caa770a871f', 'request_id': 'req-caba88fa-5006-4bb3-bac3-a4515fcf5938'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:55.343559 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-caba88fa-5006-4bb3-bac3-a4515fcf5938 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fedca4c4f14f4ed1a4492caa770a871f', 'request_id': 'req-caba88fa-5006-4bb3-bac3-a4515fcf5938'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:55.351973 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-caba88fa-5006-4bb3-bac3-a4515fcf5938 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fedca4c4f14f4ed1a4492caa770a871f', 'request_id': 'req-caba88fa-5006-4bb3-bac3-a4515fcf5938'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:55.363863 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-caba88fa-5006-4bb3-bac3-a4515fcf5938 admin admin] The token cache is being invalidate because project fedca4c4f14f4ed1a4492caa770a871f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:27:55.364153 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-caba88fa-5006-4bb3-bac3-a4515fcf5938 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:55.364890 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-caba88fa-5006-4bb3-bac3-a4515fcf5938 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:55.366495 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1265/2526] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:27:55 2022] DELETE /identity/v3/projects/fedca4c4f14f4ed1a4492caa770a871f => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:27:55.370530 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25efe381-40f7-4359-bf45-f9fbe81fed38 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:55.380610 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25efe381-40f7-4359-bf45-f9fbe81fed38 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:55.442401 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebab5f2-f853-42f7-8950-e005f81e23bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:55.443290 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebab5f2-f853-42f7-8950-e005f81e23bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:55.443636 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebab5f2-f853-42f7-8950-e005f81e23bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:55.443920 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebab5f2-f853-42f7-8950-e005f81e23bf admin admin] PATH_INFO: `/v3/users/f7531d5a1a1141a08aad242ddd2fe802` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:55.445266 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebab5f2-f853-42f7-8950-e005f81e23bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7531d5a1a1141a08aad242ddd2fe802)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:55.447190 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebab5f2-f853-42f7-8950-e005f81e23bf admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:55.502747 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4ebab5f2-f853-42f7-8950-e005f81e23bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7531d5a1a1141a08aad242ddd2fe802'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:55.503332 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4ebab5f2-f853-42f7-8950-e005f81e23bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7531d5a1a1141a08aad242ddd2fe802'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:55.517660 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4ebab5f2-f853-42f7-8950-e005f81e23bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7531d5a1a1141a08aad242ddd2fe802'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:55.525519 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4ebab5f2-f853-42f7-8950-e005f81e23bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7531d5a1a1141a08aad242ddd2fe802'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:55.534789 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4ebab5f2-f853-42f7-8950-e005f81e23bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7531d5a1a1141a08aad242ddd2fe802'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:55.542507 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1262/2527] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:27:55 2022] DELETE /identity/v3/users/f7531d5a1a1141a08aad242ddd2fe802 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:27:55.839733 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba88fa-5006-4bb3-bac3-a4515fcf5938 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:55.841391 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-caba88fa-5006-4bb3-bac3-a4515fcf5938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:55.855848 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba88fa-5006-4bb3-bac3-a4515fcf5938 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:55.924301 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb1c2f-f3e2-4e87-ae3d-932a7b3a4689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:55.925241 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb1c2f-f3e2-4e87-ae3d-932a7b3a4689 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:55.925530 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb1c2f-f3e2-4e87-ae3d-932a7b3a4689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:55.925818 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb1c2f-f3e2-4e87-ae3d-932a7b3a4689 admin admin] PATH_INFO: `/v3/projects/e30a7e8182154a6e88d588c9dc277f44` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:55.927088 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb1c2f-f3e2-4e87-ae3d-932a7b3a4689 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e30a7e8182154a6e88d588c9dc277f44)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:55.929042 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdb1c2f-f3e2-4e87-ae3d-932a7b3a4689 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:55.940744 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:27:55.981023 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-ffdb1c2f-f3e2-4e87-ae3d-932a7b3a4689 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e30a7e8182154a6e88d588c9dc277f44', 'request_id': 'req-ffdb1c2f-f3e2-4e87-ae3d-932a7b3a4689'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:55.987465 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-ffdb1c2f-f3e2-4e87-ae3d-932a7b3a4689 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e30a7e8182154a6e88d588c9dc277f44', 'request_id': 'req-ffdb1c2f-f3e2-4e87-ae3d-932a7b3a4689'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:55.996228 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-ffdb1c2f-f3e2-4e87-ae3d-932a7b3a4689 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e30a7e8182154a6e88d588c9dc277f44', 'request_id': 'req-ffdb1c2f-f3e2-4e87-ae3d-932a7b3a4689'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:56.008703 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-ffdb1c2f-f3e2-4e87-ae3d-932a7b3a4689 admin admin] The token cache is being invalidate because project e30a7e8182154a6e88d588c9dc277f44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:27:56.009075 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-ffdb1c2f-f3e2-4e87-ae3d-932a7b3a4689 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:56.009859 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-ffdb1c2f-f3e2-4e87-ae3d-932a7b3a4689 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:56.011561 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1266/2528] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:27:55 2022] DELETE /identity/v3/projects/e30a7e8182154a6e88d588c9dc277f44 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:27:56.017371 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea359b3-d6d9-4cbe-990b-1151a5b37460 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:56.017957 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea359b3-d6d9-4cbe-990b-1151a5b37460 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:56.018241 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea359b3-d6d9-4cbe-990b-1151a5b37460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:56.025985 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:27:56.039153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-0ea359b3-d6d9-4cbe-990b-1151a5b37460 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:27:56.040953 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0ea359b3-d6d9-4cbe-990b-1151a5b37460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:56.102365 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1263/2529] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:27:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:27:56.110067 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e971753-09e9-42d2-944b-c0edd506c85f None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:56.110356 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e971753-09e9-42d2-944b-c0edd506c85f None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:56.110632 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e971753-09e9-42d2-944b-c0edd506c85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:56.129840 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-8e971753-09e9-42d2-944b-c0edd506c85f None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:27:56.131077 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-8e971753-09e9-42d2-944b-c0edd506c85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:56.140021 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1267/2530] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:27:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:27:56.144192 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea359b3-d6d9-4cbe-990b-1151a5b37460 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:56.154317 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea359b3-d6d9-4cbe-990b-1151a5b37460 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:56.159143 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14a0c1-f816-4c98-bd75-a1f40d452f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:56.159834 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14a0c1-f816-4c98-bd75-a1f40d452f8b admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:56.160120 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14a0c1-f816-4c98-bd75-a1f40d452f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:56.160398 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14a0c1-f816-4c98-bd75-a1f40d452f8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:56.161126 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc14a0c1-f816-4c98-bd75-a1f40d452f8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:27:56.161433 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc14a0c1-f816-4c98-bd75-a1f40d452f8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:56.163036 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc14a0c1-f816-4c98-bd75-a1f40d452f8b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:56.175777 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1264/2531] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:27:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:27:56.180225 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e971753-09e9-42d2-944b-c0edd506c85f None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:56.190463 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e971753-09e9-42d2-944b-c0edd506c85f None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:56.195251 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e28e10-ba71-4cb6-8077-737f20838531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:56.195949 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e28e10-ba71-4cb6-8077-737f20838531 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:56.196236 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e28e10-ba71-4cb6-8077-737f20838531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:56.196512 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e28e10-ba71-4cb6-8077-737f20838531 admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:56.197219 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e28e10-ba71-4cb6-8077-737f20838531 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:56.199156 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e28e10-ba71-4cb6-8077-737f20838531 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:56.226515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1268/2532] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:27:56 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:27:56.231162 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14a0c1-f816-4c98-bd75-a1f40d452f8b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:56.241728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14a0c1-f816-4c98-bd75-a1f40d452f8b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:56.297362 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4248b766-4dbf-498f-b048-dbb13eb83e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:56.297915 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248b766-4dbf-498f-b048-dbb13eb83e5e admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:56.298194 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248b766-4dbf-498f-b048-dbb13eb83e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:56.298467 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248b766-4dbf-498f-b048-dbb13eb83e5e admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:56.299158 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4248b766-4dbf-498f-b048-dbb13eb83e5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:56.301094 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4248b766-4dbf-498f-b048-dbb13eb83e5e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:56.322544 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1265/2533] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:27:56 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:27:56.327191 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e28e10-ba71-4cb6-8077-737f20838531 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:56.338323 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e28e10-ba71-4cb6-8077-737f20838531 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:56.343131 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c85c89-235d-41a8-a99a-aaa38f4c17c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:56.343837 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c85c89-235d-41a8-a99a-aaa38f4c17c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:56.344145 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c85c89-235d-41a8-a99a-aaa38f4c17c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:56.344426 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c85c89-235d-41a8-a99a-aaa38f4c17c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:56.345189 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c85c89-235d-41a8-a99a-aaa38f4c17c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:56.346801 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c85c89-235d-41a8-a99a-aaa38f4c17c9 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:56.361069 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1269/2534] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:27:56.365217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4248b766-4dbf-498f-b048-dbb13eb83e5e admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:56.375784 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4248b766-4dbf-498f-b048-dbb13eb83e5e admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:56.380354 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e278cbce-82bb-41e5-891b-b6e500534873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:56.380999 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278cbce-82bb-41e5-891b-b6e500534873 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:56.381279 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278cbce-82bb-41e5-891b-b6e500534873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:56.381560 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278cbce-82bb-41e5-891b-b6e500534873 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:56.382269 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e278cbce-82bb-41e5-891b-b6e500534873 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:56.383909 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e278cbce-82bb-41e5-891b-b6e500534873 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:56.393715 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1266/2535] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:27:56.397794 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c85c89-235d-41a8-a99a-aaa38f4c17c9 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:56.408134 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c85c89-235d-41a8-a99a-aaa38f4c17c9 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:56.412813 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b797a27-3d8e-4074-bab0-e6f6a2596c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:56.413693 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b797a27-3d8e-4074-bab0-e6f6a2596c10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:56.413978 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b797a27-3d8e-4074-bab0-e6f6a2596c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:56.414257 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b797a27-3d8e-4074-bab0-e6f6a2596c10 admin admin] PATH_INFO: `/v3/projects/f9a05bf0016247b68184e4fb898b1631/users/8073c2364f854fb6b655bac64a93dcec/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:56.429961 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b797a27-3d8e-4074-bab0-e6f6a2596c10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9a05bf0016247b68184e4fb898b1631, user_id=8073c2364f854fb6b655bac64a93dcec, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:56.432651 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b797a27-3d8e-4074-bab0-e6f6a2596c10 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:56.446606 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1270/2536] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:27:56 2022] PUT /identity/v3/projects/f9a05bf0016247b68184e4fb898b1631/users/8073c2364f854fb6b655bac64a93dcec/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:27:56.458428 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf9168-dea0-44b9-a0fe-3f6404734336 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:56.458727 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf9168-dea0-44b9-a0fe-3f6404734336 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:56.459008 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf9168-dea0-44b9-a0fe-3f6404734336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:56.486861 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-59cf9168-dea0-44b9-a0fe-3f6404734336 None None] MFA Rules not processed for user `8073c2364f854fb6b655bac64a93dcec`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:27:56.488162 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-59cf9168-dea0-44b9-a0fe-3f6404734336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:56.567953 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:27:56.577696 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1267/2537] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:27:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:27:56.579218 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b797a27-3d8e-4074-bab0-e6f6a2596c10 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:56.654646 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b797a27-3d8e-4074-bab0-e6f6a2596c10 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:56.659679 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12972156-647d-4270-87ee-0e4b0f58c720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:56.660357 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12972156-647d-4270-87ee-0e4b0f58c720 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:56.660643 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12972156-647d-4270-87ee-0e4b0f58c720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:56.660925 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12972156-647d-4270-87ee-0e4b0f58c720 admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:56.661617 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12972156-647d-4270-87ee-0e4b0f58c720 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:56.663556 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12972156-647d-4270-87ee-0e4b0f58c720 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:56.689307 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1271/2538] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:27:56 2022] POST /identity/v3/projects => generated 387 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:27:56.693806 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf9168-dea0-44b9-a0fe-3f6404734336 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:56.704914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf9168-dea0-44b9-a0fe-3f6404734336 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:56.760835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c0f55-4249-4ac2-b0d3-f6cfd858549a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:56.761496 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c0f55-4249-4ac2-b0d3-f6cfd858549a admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:56.761783 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c0f55-4249-4ac2-b0d3-f6cfd858549a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:56.762062 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c0f55-4249-4ac2-b0d3-f6cfd858549a admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:56.762774 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c0f55-4249-4ac2-b0d3-f6cfd858549a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:56.764785 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c0f55-4249-4ac2-b0d3-f6cfd858549a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:56.785183 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1268/2539] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:27:56 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:27:56.789219 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12972156-647d-4270-87ee-0e4b0f58c720 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:56.800403 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12972156-647d-4270-87ee-0e4b0f58c720 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:56.805010 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e20b16-f731-4744-839c-8d7ea05116fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:56.805597 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e20b16-f731-4744-839c-8d7ea05116fc admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:56.805876 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e20b16-f731-4744-839c-8d7ea05116fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:56.806147 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e20b16-f731-4744-839c-8d7ea05116fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:56.806862 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e20b16-f731-4744-839c-8d7ea05116fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:56.808497 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e20b16-f731-4744-839c-8d7ea05116fc admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:56.818898 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1272/2540] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:27:56.823139 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c0f55-4249-4ac2-b0d3-f6cfd858549a admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:56.833504 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c0f55-4249-4ac2-b0d3-f6cfd858549a admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:56.838216 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e609f-df9e-4777-8623-5f5458393d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:56.839114 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e609f-df9e-4777-8623-5f5458393d37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:56.839376 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e609f-df9e-4777-8623-5f5458393d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:56.839800 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e609f-df9e-4777-8623-5f5458393d37 admin admin] PATH_INFO: `/v3/projects/b32d71703b454d9ba093f56af39963bc/users/758a83c72dfa4e8cbbb8ff10b2b43b7a/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:56.855655 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e609f-df9e-4777-8623-5f5458393d37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b32d71703b454d9ba093f56af39963bc, user_id=758a83c72dfa4e8cbbb8ff10b2b43b7a, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:56.859315 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e609f-df9e-4777-8623-5f5458393d37 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:56.873058 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1269/2541] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:27:56 2022] PUT /identity/v3/projects/b32d71703b454d9ba093f56af39963bc/users/758a83c72dfa4e8cbbb8ff10b2b43b7a/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:27:56.877391 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e20b16-f731-4744-839c-8d7ea05116fc admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:56.888833 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e20b16-f731-4744-839c-8d7ea05116fc admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:56.894525 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:27:56.953317 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e72331-d1d0-4763-a953-e5f0215f6b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:56.954022 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e72331-d1d0-4763-a953-e5f0215f6b38 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:56.954311 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e72331-d1d0-4763-a953-e5f0215f6b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:56.954842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e72331-d1d0-4763-a953-e5f0215f6b38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:56.955316 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e72331-d1d0-4763-a953-e5f0215f6b38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:56.957003 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e72331-d1d0-4763-a953-e5f0215f6b38 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:56.968187 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1273/2542] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:27:56.972447 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e609f-df9e-4777-8623-5f5458393d37 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:56.983078 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e609f-df9e-4777-8623-5f5458393d37 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:56.988939 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036bd7a-a095-4f1a-97a1-8f4a4898cab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:56.990682 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036bd7a-a095-4f1a-97a1-8f4a4898cab2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:56.991353 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036bd7a-a095-4f1a-97a1-8f4a4898cab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:56.991678 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036bd7a-a095-4f1a-97a1-8f4a4898cab2 admin admin] PATH_INFO: `/v3/projects/b32d71703b454d9ba093f56af39963bc/users/758a83c72dfa4e8cbbb8ff10b2b43b7a/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:56.993712 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c036bd7a-a095-4f1a-97a1-8f4a4898cab2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b32d71703b454d9ba093f56af39963bc, user_id=758a83c72dfa4e8cbbb8ff10b2b43b7a, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:56.996404 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c036bd7a-a095-4f1a-97a1-8f4a4898cab2 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:57.006821 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1270/2543] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:27:56 2022] PUT /identity/v3/projects/b32d71703b454d9ba093f56af39963bc/users/758a83c72dfa4e8cbbb8ff10b2b43b7a/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:27:57.017893 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fef8c5-3a34-4f10-b3a9-45188a78f799 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:57.018187 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fef8c5-3a34-4f10-b3a9-45188a78f799 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:57.018468 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fef8c5-3a34-4f10-b3a9-45188a78f799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:57.045207 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-93fef8c5-3a34-4f10-b3a9-45188a78f799 None None] MFA Rules not processed for user `758a83c72dfa4e8cbbb8ff10b2b43b7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:27:57.046418 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-93fef8c5-3a34-4f10-b3a9-45188a78f799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:57.115878 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1274/2544] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:27:57 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:27:57.127552 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036bd7a-a095-4f1a-97a1-8f4a4898cab2 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:57.129399 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c036bd7a-a095-4f1a-97a1-8f4a4898cab2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:57.190356 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036bd7a-a095-4f1a-97a1-8f4a4898cab2 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:57.194332 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4fb456-cea8-4e4d-ae46-5a0d4fe95ad3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:57.194986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4fb456-cea8-4e4d-ae46-5a0d4fe95ad3 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:57.195269 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4fb456-cea8-4e4d-ae46-5a0d4fe95ad3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:57.195561 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4fb456-cea8-4e4d-ae46-5a0d4fe95ad3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:57.208583 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4fb456-cea8-4e4d-ae46-5a0d4fe95ad3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:57.210017 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4fb456-cea8-4e4d-ae46-5a0d4fe95ad3 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:57.220591 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1271/2545] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:27:57 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:27:57.283886 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fef8c5-3a34-4f10-b3a9-45188a78f799 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:57.345036 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fef8c5-3a34-4f10-b3a9-45188a78f799 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:57.349444 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa5a041-0a1f-4e3c-bd9e-651153b1e0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:57.350305 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5a041-0a1f-4e3c-bd9e-651153b1e0b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:57.350584 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5a041-0a1f-4e3c-bd9e-651153b1e0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:57.350855 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5a041-0a1f-4e3c-bd9e-651153b1e0b6 admin admin] PATH_INFO: `/v3/users/8073c2364f854fb6b655bac64a93dcec` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:57.352059 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa5a041-0a1f-4e3c-bd9e-651153b1e0b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8073c2364f854fb6b655bac64a93dcec)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:57.353972 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa5a041-0a1f-4e3c-bd9e-651153b1e0b6 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:57.391223 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:27:57.405120 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-bfa5a041-0a1f-4e3c-bd9e-651153b1e0b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8073c2364f854fb6b655bac64a93dcec'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:57.405965 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-bfa5a041-0a1f-4e3c-bd9e-651153b1e0b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8073c2364f854fb6b655bac64a93dcec'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:57.417958 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-bfa5a041-0a1f-4e3c-bd9e-651153b1e0b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8073c2364f854fb6b655bac64a93dcec'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:57.424965 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-bfa5a041-0a1f-4e3c-bd9e-651153b1e0b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8073c2364f854fb6b655bac64a93dcec'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:57.425650 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-bfa5a041-0a1f-4e3c-bd9e-651153b1e0b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8073c2364f854fb6b655bac64a93dcec'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:57.439555 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1275/2546] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:27:57 2022] DELETE /identity/v3/users/8073c2364f854fb6b655bac64a93dcec => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:27:57.444155 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159dba37-3b20-48ad-9590-af9e2cb76086 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:57.444489 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159dba37-3b20-48ad-9590-af9e2cb76086 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:57.444768 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159dba37-3b20-48ad-9590-af9e2cb76086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:57.463427 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-159dba37-3b20-48ad-9590-af9e2cb76086 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:27:57.464583 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-159dba37-3b20-48ad-9590-af9e2cb76086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:57.514486 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:27:57.529786 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1272/2547] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:27:57.539375 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa5a041-0a1f-4e3c-bd9e-651153b1e0b6 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:57.540946 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-bfa5a041-0a1f-4e3c-bd9e-651153b1e0b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:57.582767 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159dba37-3b20-48ad-9590-af9e2cb76086 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:57.584215 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-159dba37-3b20-48ad-9590-af9e2cb76086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:57.610295 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa5a041-0a1f-4e3c-bd9e-651153b1e0b6 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:57.614128 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b53229-2d6e-4e54-9d5f-9a281a60fdc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:57.614805 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b53229-2d6e-4e54-9d5f-9a281a60fdc4 service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:57.615129 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b53229-2d6e-4e54-9d5f-9a281a60fdc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:57.615413 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b53229-2d6e-4e54-9d5f-9a281a60fdc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:57.626089 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b53229-2d6e-4e54-9d5f-9a281a60fdc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:57.627601 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b53229-2d6e-4e54-9d5f-9a281a60fdc4 service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:57.638938 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1276/2548] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:27:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:27:57.685533 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159dba37-3b20-48ad-9590-af9e2cb76086 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:57.690808 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b8d5d-58f9-41c2-af63-9a319c8ce1f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:57.692183 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b8d5d-58f9-41c2-af63-9a319c8ce1f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:57.692473 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b8d5d-58f9-41c2-af63-9a319c8ce1f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:57.692755 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b8d5d-58f9-41c2-af63-9a319c8ce1f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:57.695684 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b23b8d5d-58f9-41c2-af63-9a319c8ce1f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:57.821589 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23b8d5d-58f9-41c2-af63-9a319c8ce1f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:57.823397 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23b8d5d-58f9-41c2-af63-9a319c8ce1f7 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:57.851372 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1273/2549] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:27:57 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:27:58.100251 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b53229-2d6e-4e54-9d5f-9a281a60fdc4 service neutron] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:58.101814 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-b2b53229-2d6e-4e54-9d5f-9a281a60fdc4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:58.128609 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b53229-2d6e-4e54-9d5f-9a281a60fdc4 service neutron] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:58.135050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521abb09-4253-40e8-9b3f-a87a87c9929e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:58.136236 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521abb09-4253-40e8-9b3f-a87a87c9929e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:58.136688 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521abb09-4253-40e8-9b3f-a87a87c9929e admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:58.137174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521abb09-4253-40e8-9b3f-a87a87c9929e admin admin] PATH_INFO: `/v3/projects/f9a05bf0016247b68184e4fb898b1631` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:58.139220 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521abb09-4253-40e8-9b3f-a87a87c9929e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9a05bf0016247b68184e4fb898b1631)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:58.141742 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521abb09-4253-40e8-9b3f-a87a87c9929e admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:58.238539 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-521abb09-4253-40e8-9b3f-a87a87c9929e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9a05bf0016247b68184e4fb898b1631', 'request_id': 'req-521abb09-4253-40e8-9b3f-a87a87c9929e'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:58.247662 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-521abb09-4253-40e8-9b3f-a87a87c9929e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9a05bf0016247b68184e4fb898b1631', 'request_id': 'req-521abb09-4253-40e8-9b3f-a87a87c9929e'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:58.263700 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-521abb09-4253-40e8-9b3f-a87a87c9929e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9a05bf0016247b68184e4fb898b1631', 'request_id': 'req-521abb09-4253-40e8-9b3f-a87a87c9929e'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:58.279772 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-521abb09-4253-40e8-9b3f-a87a87c9929e admin admin] The token cache is being invalidate because project f9a05bf0016247b68184e4fb898b1631 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:27:58.279772 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-521abb09-4253-40e8-9b3f-a87a87c9929e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:58.280549 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-521abb09-4253-40e8-9b3f-a87a87c9929e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:58.282950 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1277/2550] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:27:58 2022] DELETE /identity/v3/projects/f9a05bf0016247b68184e4fb898b1631 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:27:58.286725 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b8d5d-58f9-41c2-af63-9a319c8ce1f7 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:58.302225 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b8d5d-58f9-41c2-af63-9a319c8ce1f7 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:58.375284 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f08bca-a123-45e4-9d75-692e43bf559f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:58.376908 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f08bca-a123-45e4-9d75-692e43bf559f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:58.377738 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f08bca-a123-45e4-9d75-692e43bf559f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:58.377738 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f08bca-a123-45e4-9d75-692e43bf559f admin admin] PATH_INFO: `/v3/users/758a83c72dfa4e8cbbb8ff10b2b43b7a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:58.378930 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f08bca-a123-45e4-9d75-692e43bf559f admin admin] RBAC: Authorizing `identity:delete_user(user_id=758a83c72dfa4e8cbbb8ff10b2b43b7a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:58.380983 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f08bca-a123-45e4-9d75-692e43bf559f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:58.416063 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:27:58.476006 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-15f08bca-a123-45e4-9d75-692e43bf559f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '758a83c72dfa4e8cbbb8ff10b2b43b7a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:58.476928 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-15f08bca-a123-45e4-9d75-692e43bf559f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '758a83c72dfa4e8cbbb8ff10b2b43b7a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:58.498160 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-15f08bca-a123-45e4-9d75-692e43bf559f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '758a83c72dfa4e8cbbb8ff10b2b43b7a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:58.506028 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-15f08bca-a123-45e4-9d75-692e43bf559f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '758a83c72dfa4e8cbbb8ff10b2b43b7a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:58.518848 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-15f08bca-a123-45e4-9d75-692e43bf559f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '758a83c72dfa4e8cbbb8ff10b2b43b7a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:58.528170 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1274/2551] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:27:58 2022] DELETE /identity/v3/users/758a83c72dfa4e8cbbb8ff10b2b43b7a => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:27:58.849701 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521abb09-4253-40e8-9b3f-a87a87c9929e admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:58.851949 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-521abb09-4253-40e8-9b3f-a87a87c9929e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:58.882322 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521abb09-4253-40e8-9b3f-a87a87c9929e admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:58.928406 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:27:58.967903 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c3d15a-1b09-4f44-8747-39c3cf96c9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:58.969183 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3d15a-1b09-4f44-8747-39c3cf96c9ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:58.969535 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3d15a-1b09-4f44-8747-39c3cf96c9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:58.973661 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3d15a-1b09-4f44-8747-39c3cf96c9ce admin admin] PATH_INFO: `/v3/projects/b32d71703b454d9ba093f56af39963bc` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:58.975975 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c3d15a-1b09-4f44-8747-39c3cf96c9ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=b32d71703b454d9ba093f56af39963bc)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:58.978336 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c3d15a-1b09-4f44-8747-39c3cf96c9ce admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:59.051874 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-93c3d15a-1b09-4f44-8747-39c3cf96c9ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b32d71703b454d9ba093f56af39963bc', 'request_id': 'req-93c3d15a-1b09-4f44-8747-39c3cf96c9ce'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:59.058780 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-93c3d15a-1b09-4f44-8747-39c3cf96c9ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b32d71703b454d9ba093f56af39963bc', 'request_id': 'req-93c3d15a-1b09-4f44-8747-39c3cf96c9ce'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:59.068192 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-93c3d15a-1b09-4f44-8747-39c3cf96c9ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b32d71703b454d9ba093f56af39963bc', 'request_id': 'req-93c3d15a-1b09-4f44-8747-39c3cf96c9ce'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:59.081076 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-93c3d15a-1b09-4f44-8747-39c3cf96c9ce admin admin] The token cache is being invalidate because project b32d71703b454d9ba093f56af39963bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:27:59.081366 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-93c3d15a-1b09-4f44-8747-39c3cf96c9ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:59.082028 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-93c3d15a-1b09-4f44-8747-39c3cf96c9ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:27:59.083734 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1278/2552] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:27:58 2022] DELETE /identity/v3/projects/b32d71703b454d9ba093f56af39963bc => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:27:59.088878 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be071d70-c8cf-4299-9c1d-873e9d5d1f7c None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:59.089370 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be071d70-c8cf-4299-9c1d-873e9d5d1f7c None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:59.089633 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be071d70-c8cf-4299-9c1d-873e9d5d1f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:59.112702 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-be071d70-c8cf-4299-9c1d-873e9d5d1f7c None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:27:59.113686 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-be071d70-c8cf-4299-9c1d-873e9d5d1f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:59.199703 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1275/2553] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:27:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:27:59.201775 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6864e-889b-4e0c-9de1-fa73ab30117e None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:59.202207 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6864e-889b-4e0c-9de1-fa73ab30117e None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:59.202512 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6864e-889b-4e0c-9de1-fa73ab30117e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:59.222479 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-71f6864e-889b-4e0c-9de1-fa73ab30117e None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:27:59.223921 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-71f6864e-889b-4e0c-9de1-fa73ab30117e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:59.233754 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1279/2554] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:27:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:27:59.238278 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be071d70-c8cf-4299-9c1d-873e9d5d1f7c None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:59.249983 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be071d70-c8cf-4299-9c1d-873e9d5d1f7c None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:59.255466 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3658534c-c3bb-49bf-8062-415a14ef6829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:59.256279 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658534c-c3bb-49bf-8062-415a14ef6829 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:59.260075 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658534c-c3bb-49bf-8062-415a14ef6829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:59.260310 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658534c-c3bb-49bf-8062-415a14ef6829 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:59.260953 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3658534c-c3bb-49bf-8062-415a14ef6829 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:27:59.261591 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3658534c-c3bb-49bf-8062-415a14ef6829 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:59.263789 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3658534c-c3bb-49bf-8062-415a14ef6829 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:59.283588 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1276/2555] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:27:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:27:59.287958 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f6864e-889b-4e0c-9de1-fa73ab30117e None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:59.297626 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f6864e-889b-4e0c-9de1-fa73ab30117e None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:59.302180 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c937a-efdc-4e55-ad9b-3e61d4e9d985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:59.302821 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c937a-efdc-4e55-ad9b-3e61d4e9d985 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:59.303099 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c937a-efdc-4e55-ad9b-3e61d4e9d985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:59.303368 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02c937a-efdc-4e55-ad9b-3e61d4e9d985 admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:59.304113 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02c937a-efdc-4e55-ad9b-3e61d4e9d985 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:59.306051 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02c937a-efdc-4e55-ad9b-3e61d4e9d985 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:59.336074 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1280/2556] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:27:59 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:27:59.337299 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3658534c-c3bb-49bf-8062-415a14ef6829 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:59.347969 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3658534c-c3bb-49bf-8062-415a14ef6829 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:59.409473 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccaac7c-6368-4edd-a661-bebb57091daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:59.410228 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccaac7c-6368-4edd-a661-bebb57091daf admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:59.410490 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccaac7c-6368-4edd-a661-bebb57091daf admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:59.410764 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccaac7c-6368-4edd-a661-bebb57091daf admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:59.411765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccaac7c-6368-4edd-a661-bebb57091daf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:59.413894 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccaac7c-6368-4edd-a661-bebb57091daf admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:59.439197 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1277/2557] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:27:59 2022] POST /identity/v3/users => generated 284 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:27:59.441512 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c937a-efdc-4e55-ad9b-3e61d4e9d985 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:59.458071 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02c937a-efdc-4e55-ad9b-3e61d4e9d985 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:59.464673 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c48077-c838-44b8-b7ed-e651ebaf407f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:59.466297 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c48077-c838-44b8-b7ed-e651ebaf407f admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:59.466771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c48077-c838-44b8-b7ed-e651ebaf407f admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:59.467210 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c48077-c838-44b8-b7ed-e651ebaf407f admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:59.468306 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c48077-c838-44b8-b7ed-e651ebaf407f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:59.470470 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c48077-c838-44b8-b7ed-e651ebaf407f admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:59.485821 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1281/2558] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:27:59 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:27:59.489884 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccaac7c-6368-4edd-a661-bebb57091daf admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:59.503987 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccaac7c-6368-4edd-a661-bebb57091daf admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:59.512232 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d64a57d-86a2-4615-9b60-d6b7bd799177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:59.513567 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d64a57d-86a2-4615-9b60-d6b7bd799177 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:59.514010 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d64a57d-86a2-4615-9b60-d6b7bd799177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:59.514233 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d64a57d-86a2-4615-9b60-d6b7bd799177 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:59.515038 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d64a57d-86a2-4615-9b60-d6b7bd799177 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:59.518790 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d64a57d-86a2-4615-9b60-d6b7bd799177 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:59.532676 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1278/2559] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:27:59 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:27:59.536495 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c48077-c838-44b8-b7ed-e651ebaf407f admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:59.564386 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c48077-c838-44b8-b7ed-e651ebaf407f admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:59.578972 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627c374f-889c-4fd7-9cd1-297c47e40097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:59.580278 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c374f-889c-4fd7-9cd1-297c47e40097 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:59.580658 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c374f-889c-4fd7-9cd1-297c47e40097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:59.581030 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c374f-889c-4fd7-9cd1-297c47e40097 admin admin] PATH_INFO: `/v3/projects/3679cf44b5044fe5b483d3008f85e65c/users/9d07325437f64e5f8069292e5a713b23/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:59.598783 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627c374f-889c-4fd7-9cd1-297c47e40097 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3679cf44b5044fe5b483d3008f85e65c, user_id=9d07325437f64e5f8069292e5a713b23, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:59.601995 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627c374f-889c-4fd7-9cd1-297c47e40097 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:59.632140 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1282/2560] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:27:59 2022] PUT /identity/v3/projects/3679cf44b5044fe5b483d3008f85e65c/users/9d07325437f64e5f8069292e5a713b23/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:27:59.633599 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c8297-565a-4cc1-a6ab-cd1a90ad8d2e None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:59.633895 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c8297-565a-4cc1-a6ab-cd1a90ad8d2e None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:59.634179 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c8297-565a-4cc1-a6ab-cd1a90ad8d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:59.649425 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:27:59.670320 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627c374f-889c-4fd7-9cd1-297c47e40097 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:59.672750 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-627c374f-889c-4fd7-9cd1-297c47e40097 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:59.679441 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-1d3c8297-565a-4cc1-a6ab-cd1a90ad8d2e None None] MFA Rules not processed for user `9d07325437f64e5f8069292e5a713b23`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:27:59.681366 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1d3c8297-565a-4cc1-a6ab-cd1a90ad8d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:59.778063 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:27:59.783686 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1279/2561] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:27:59 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:27:59.788248 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c8297-565a-4cc1-a6ab-cd1a90ad8d2e None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:59.808398 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627c374f-889c-4fd7-9cd1-297c47e40097 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:59.817796 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca50a0-2986-4dc9-95d2-72ab270f6416 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:59.818471 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca50a0-2986-4dc9-95d2-72ab270f6416 service nova] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:59.818754 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca50a0-2986-4dc9-95d2-72ab270f6416 service nova] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:59.819027 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca50a0-2986-4dc9-95d2-72ab270f6416 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:59.821039 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-30ca50a0-2986-4dc9-95d2-72ab270f6416 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:27:59.838945 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ca50a0-2986-4dc9-95d2-72ab270f6416 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:59.841460 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ca50a0-2986-4dc9-95d2-72ab270f6416 service nova] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:59.859711 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1283/2562] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:27:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:27:59.867176 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c8297-565a-4cc1-a6ab-cd1a90ad8d2e None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:27:59.875211 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a3490a-a858-4894-aa8b-83f19fbc76f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:27:59.876509 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3490a-a858-4894-aa8b-83f19fbc76f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:27:59.877140 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3490a-a858-4894-aa8b-83f19fbc76f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:27:59.877689 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3490a-a858-4894-aa8b-83f19fbc76f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:27:59.878588 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a3490a-a858-4894-aa8b-83f19fbc76f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:27:59.882972 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a3490a-a858-4894-aa8b-83f19fbc76f9 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:27:59.915643 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1280/2563] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:27:59 2022] POST /identity/v3/projects => generated 367 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:27:59.917507 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca50a0-2986-4dc9-95d2-72ab270f6416 service nova] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:27:59.931145 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca50a0-2986-4dc9-95d2-72ab270f6416 service nova] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:00.003946 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7241a94-2091-4598-9376-23e40838ac2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:00.004923 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7241a94-2091-4598-9376-23e40838ac2a admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:00.005397 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7241a94-2091-4598-9376-23e40838ac2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:00.005844 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7241a94-2091-4598-9376-23e40838ac2a admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:00.006712 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7241a94-2091-4598-9376-23e40838ac2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:00.008746 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7241a94-2091-4598-9376-23e40838ac2a admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:00.033285 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1284/2564] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:27:59 2022] POST /identity/v3/users => generated 290 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:28:00.034512 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a3490a-a858-4894-aa8b-83f19fbc76f9 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:00.045897 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a3490a-a858-4894-aa8b-83f19fbc76f9 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:00.051226 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9d1149-fcf0-4b76-ba34-30f2f2f35816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:00.052118 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d1149-fcf0-4b76-ba34-30f2f2f35816 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:00.052401 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d1149-fcf0-4b76-ba34-30f2f2f35816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:00.052705 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d1149-fcf0-4b76-ba34-30f2f2f35816 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:00.053520 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9d1149-fcf0-4b76-ba34-30f2f2f35816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:00.055131 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9d1149-fcf0-4b76-ba34-30f2f2f35816 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:00.067104 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1281/2565] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:28:00.071318 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7241a94-2091-4598-9376-23e40838ac2a admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:00.081474 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7241a94-2091-4598-9376-23e40838ac2a admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:00.086057 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1730e72b-eafa-4efb-8f87-2ae34d4c7ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:00.086938 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1730e72b-eafa-4efb-8f87-2ae34d4c7ac2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:00.087217 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1730e72b-eafa-4efb-8f87-2ae34d4c7ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:00.087492 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1730e72b-eafa-4efb-8f87-2ae34d4c7ac2 admin admin] PATH_INFO: `/v3/projects/84a50f629d344241be76e3d5a11af9f6/users/fc5aa39fb1144552a519b20596845924/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:00.103648 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1730e72b-eafa-4efb-8f87-2ae34d4c7ac2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84a50f629d344241be76e3d5a11af9f6, user_id=fc5aa39fb1144552a519b20596845924, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:00.106903 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1730e72b-eafa-4efb-8f87-2ae34d4c7ac2 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:00.121590 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1285/2566] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:28:00 2022] PUT /identity/v3/projects/84a50f629d344241be76e3d5a11af9f6/users/fc5aa39fb1144552a519b20596845924/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:00.126149 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9d1149-fcf0-4b76-ba34-30f2f2f35816 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:00.136389 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9d1149-fcf0-4b76-ba34-30f2f2f35816 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:00.191607 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e5c3b8-96fc-42ac-9bf9-ac98cd6abe2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:00.192264 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5c3b8-96fc-42ac-9bf9-ac98cd6abe2e admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:00.192541 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5c3b8-96fc-42ac-9bf9-ac98cd6abe2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:00.192812 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5c3b8-96fc-42ac-9bf9-ac98cd6abe2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:00.193524 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e5c3b8-96fc-42ac-9bf9-ac98cd6abe2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:00.195129 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e5c3b8-96fc-42ac-9bf9-ac98cd6abe2e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:00.204699 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1282/2567] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:00 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:28:00.209033 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1730e72b-eafa-4efb-8f87-2ae34d4c7ac2 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:00.219255 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1730e72b-eafa-4efb-8f87-2ae34d4c7ac2 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:00.223796 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9917236-b2d9-4fa9-b638-abbefead1914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:00.224669 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9917236-b2d9-4fa9-b638-abbefead1914 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:00.224949 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9917236-b2d9-4fa9-b638-abbefead1914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:00.225222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9917236-b2d9-4fa9-b638-abbefead1914 admin admin] PATH_INFO: `/v3/projects/84a50f629d344241be76e3d5a11af9f6/users/fc5aa39fb1144552a519b20596845924/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:00.227201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9917236-b2d9-4fa9-b638-abbefead1914 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84a50f629d344241be76e3d5a11af9f6, user_id=fc5aa39fb1144552a519b20596845924, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:00.230671 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9917236-b2d9-4fa9-b638-abbefead1914 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:00.242925 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1286/2568] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:28:00 2022] PUT /identity/v3/projects/84a50f629d344241be76e3d5a11af9f6/users/fc5aa39fb1144552a519b20596845924/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:00.254395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdaa9a-9175-42a5-a2c4-d41dc5a8a130 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:00.254684 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdaa9a-9175-42a5-a2c4-d41dc5a8a130 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:00.254991 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bdaa9a-9175-42a5-a2c4-d41dc5a8a130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:00.267483 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:00.285146 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-13bdaa9a-9175-42a5-a2c4-d41dc5a8a130 None None] MFA Rules not processed for user `fc5aa39fb1144552a519b20596845924`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:00.286409 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-13bdaa9a-9175-42a5-a2c4-d41dc5a8a130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:00.364966 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1283/2569] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:00 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:00.377475 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9917236-b2d9-4fa9-b638-abbefead1914 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:00.379346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-b9917236-b2d9-4fa9-b638-abbefead1914 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:00.447841 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9917236-b2d9-4fa9-b638-abbefead1914 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:00.451583 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4cd4bc-fc59-4fef-980c-31b2cf25956f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:00.452237 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4cd4bc-fc59-4fef-980c-31b2cf25956f service cinder] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:00.452522 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4cd4bc-fc59-4fef-980c-31b2cf25956f service cinder] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:00.452800 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4cd4bc-fc59-4fef-980c-31b2cf25956f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:00.463067 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4cd4bc-fc59-4fef-980c-31b2cf25956f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:00.464570 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4cd4bc-fc59-4fef-980c-31b2cf25956f service cinder] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:00.475081 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1287/2570] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:00 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:28:00.567992 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bdaa9a-9175-42a5-a2c4-d41dc5a8a130 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:00.578206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bdaa9a-9175-42a5-a2c4-d41dc5a8a130 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:00.583278 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611289a3-9219-4e02-99f1-90be02d2750a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:00.584102 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611289a3-9219-4e02-99f1-90be02d2750a service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:00.584516 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611289a3-9219-4e02-99f1-90be02d2750a service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:00.586339 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611289a3-9219-4e02-99f1-90be02d2750a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:00.644826 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611289a3-9219-4e02-99f1-90be02d2750a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:00.646278 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611289a3-9219-4e02-99f1-90be02d2750a service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:00.656328 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1284/2571] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:00 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:28:01.117567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4cd4bc-fc59-4fef-980c-31b2cf25956f service cinder] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:01.118970 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-ac4cd4bc-fc59-4fef-980c-31b2cf25956f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:01.169827 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:01.180832 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4cd4bc-fc59-4fef-980c-31b2cf25956f service cinder] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:01.185176 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a55454-d074-4ff3-be92-a265e9a07e3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:01.185901 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a55454-d074-4ff3-be92-a265e9a07e3b service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:01.186259 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a55454-d074-4ff3-be92-a265e9a07e3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:01.186595 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a55454-d074-4ff3-be92-a265e9a07e3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:01.188958 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-78a55454-d074-4ff3-be92-a265e9a07e3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:01.263644 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a55454-d074-4ff3-be92-a265e9a07e3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:01.265281 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a55454-d074-4ff3-be92-a265e9a07e3b service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:01.276989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1288/2572] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:28:05.813771 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611289a3-9219-4e02-99f1-90be02d2750a service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:05.825787 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611289a3-9219-4e02-99f1-90be02d2750a service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:05.831204 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a45dd3-e812-4945-a676-17f90ffba71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:05.832118 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a45dd3-e812-4945-a676-17f90ffba71e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:05.832616 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a45dd3-e812-4945-a676-17f90ffba71e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:05.832997 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a45dd3-e812-4945-a676-17f90ffba71e admin admin] PATH_INFO: `/v3/users/9d07325437f64e5f8069292e5a713b23` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:05.834463 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a45dd3-e812-4945-a676-17f90ffba71e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d07325437f64e5f8069292e5a713b23)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:05.836756 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a45dd3-e812-4945-a676-17f90ffba71e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:05.896679 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f9a45dd3-e812-4945-a676-17f90ffba71e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d07325437f64e5f8069292e5a713b23'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:05.897898 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f9a45dd3-e812-4945-a676-17f90ffba71e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d07325437f64e5f8069292e5a713b23'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:05.914665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f9a45dd3-e812-4945-a676-17f90ffba71e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d07325437f64e5f8069292e5a713b23'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:05.923390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f9a45dd3-e812-4945-a676-17f90ffba71e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d07325437f64e5f8069292e5a713b23'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:05.927924 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a55454-d074-4ff3-be92-a265e9a07e3b service neutron] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:05.932442 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-78a55454-d074-4ff3-be92-a265e9a07e3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:05.932597 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f9a45dd3-e812-4945-a676-17f90ffba71e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d07325437f64e5f8069292e5a713b23'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:05.942759 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1285/2573] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:28:05 2022] DELETE /identity/v3/users/9d07325437f64e5f8069292e5a713b23 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:05.944778 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7574cf-1f22-49cc-90f0-5d304e1a0da9 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:05.945134 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7574cf-1f22-49cc-90f0-5d304e1a0da9 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:05.945499 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7574cf-1f22-49cc-90f0-5d304e1a0da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:05.952648 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:05.973281 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-6b7574cf-1f22-49cc-90f0-5d304e1a0da9 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:05.974666 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6b7574cf-1f22-49cc-90f0-5d304e1a0da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:06.047345 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a55454-d074-4ff3-be92-a265e9a07e3b service neutron] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:06.080686 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1286/2574] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:06.081540 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7574cf-1f22-49cc-90f0-5d304e1a0da9 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:06.083453 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6b7574cf-1f22-49cc-90f0-5d304e1a0da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:06.107323 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7574cf-1f22-49cc-90f0-5d304e1a0da9 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:06.114699 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ade73c2-e82f-45dc-93be-5d6827dd8ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:06.115922 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade73c2-e82f-45dc-93be-5d6827dd8ecd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:06.116273 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade73c2-e82f-45dc-93be-5d6827dd8ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:06.116575 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade73c2-e82f-45dc-93be-5d6827dd8ecd admin admin] PATH_INFO: `/v3/users/646d135cd569428c8eda3577df418963` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:06.118711 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ade73c2-e82f-45dc-93be-5d6827dd8ecd admin admin] RBAC: Authorizing `identity:delete_user(user_id=646d135cd569428c8eda3577df418963)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:06.120850 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ade73c2-e82f-45dc-93be-5d6827dd8ecd admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:06.132463 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13373576-ddbe-4949-a06e-4130b0286102 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64824ba84a024930bbc177f8a8d9a046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:06.133269 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13373576-ddbe-4949-a06e-4130b0286102 service placement] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:06.133557 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13373576-ddbe-4949-a06e-4130b0286102 service placement] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:06.133858 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13373576-ddbe-4949-a06e-4130b0286102 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:06.139216 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-13373576-ddbe-4949-a06e-4130b0286102 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:06.156179 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13373576-ddbe-4949-a06e-4130b0286102 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:06.157824 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13373576-ddbe-4949-a06e-4130b0286102 service placement] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:06.172855 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1289/2575] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:28:06.174187 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13373576-ddbe-4949-a06e-4130b0286102 service placement] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:06.175895 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-13373576-ddbe-4949-a06e-4130b0286102 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:06.203317 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8ade73c2-e82f-45dc-93be-5d6827dd8ecd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '646d135cd569428c8eda3577df418963'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:06.204362 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8ade73c2-e82f-45dc-93be-5d6827dd8ecd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '646d135cd569428c8eda3577df418963'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:06.220732 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:06.224860 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8ade73c2-e82f-45dc-93be-5d6827dd8ecd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '646d135cd569428c8eda3577df418963'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:06.234167 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8ade73c2-e82f-45dc-93be-5d6827dd8ecd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '646d135cd569428c8eda3577df418963'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:06.241769 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8ade73c2-e82f-45dc-93be-5d6827dd8ecd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '646d135cd569428c8eda3577df418963'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:06.248874 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1287/2576] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:28:06 2022] DELETE /identity/v3/users/646d135cd569428c8eda3577df418963 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:06.260671 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13373576-ddbe-4949-a06e-4130b0286102 service placement] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:06.312403 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b0cd7-ee2e-4549-8e9d-5f7c05371020 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:06.313055 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b0cd7-ee2e-4549-8e9d-5f7c05371020 service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:06.313334 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b0cd7-ee2e-4549-8e9d-5f7c05371020 service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:06.313604 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b0cd7-ee2e-4549-8e9d-5f7c05371020 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:06.315583 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-314b0cd7-ee2e-4549-8e9d-5f7c05371020 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:06.379235 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314b0cd7-ee2e-4549-8e9d-5f7c05371020 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:06.380732 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314b0cd7-ee2e-4549-8e9d-5f7c05371020 service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:06.390232 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1290/2577] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:28:06.459913 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ade73c2-e82f-45dc-93be-5d6827dd8ecd admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:06.461641 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8ade73c2-e82f-45dc-93be-5d6827dd8ecd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:06.477708 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ade73c2-e82f-45dc-93be-5d6827dd8ecd admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:06.484328 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553a90c6-1847-4362-8822-ef624c24a817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:06.485131 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553a90c6-1847-4362-8822-ef624c24a817 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:06.485595 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553a90c6-1847-4362-8822-ef624c24a817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:06.485860 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553a90c6-1847-4362-8822-ef624c24a817 admin admin] PATH_INFO: `/v3/projects/a21be4dfe67e43d0915559bad38fa60c` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:06.487321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553a90c6-1847-4362-8822-ef624c24a817 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a21be4dfe67e43d0915559bad38fa60c)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:06.489416 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553a90c6-1847-4362-8822-ef624c24a817 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:06.564166 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-553a90c6-1847-4362-8822-ef624c24a817 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a21be4dfe67e43d0915559bad38fa60c', 'request_id': 'req-553a90c6-1847-4362-8822-ef624c24a817'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:06.573638 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-553a90c6-1847-4362-8822-ef624c24a817 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a21be4dfe67e43d0915559bad38fa60c', 'request_id': 'req-553a90c6-1847-4362-8822-ef624c24a817'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:06.583630 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-553a90c6-1847-4362-8822-ef624c24a817 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a21be4dfe67e43d0915559bad38fa60c', 'request_id': 'req-553a90c6-1847-4362-8822-ef624c24a817'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:06.592516 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:06.598502 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-553a90c6-1847-4362-8822-ef624c24a817 admin admin] The token cache is being invalidate because project a21be4dfe67e43d0915559bad38fa60c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:28:06.598502 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-553a90c6-1847-4362-8822-ef624c24a817 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:06.599476 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-553a90c6-1847-4362-8822-ef624c24a817 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:06.601643 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1288/2578] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:28:06 2022] DELETE /identity/v3/projects/a21be4dfe67e43d0915559bad38fa60c => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:06.608006 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e3d1c-c2db-46b1-92f4-502f1bfdbe6d None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:06.608581 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e3d1c-c2db-46b1-92f4-502f1bfdbe6d None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:06.608880 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e3d1c-c2db-46b1-92f4-502f1bfdbe6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:06.629857 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-5a3e3d1c-c2db-46b1-92f4-502f1bfdbe6d None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:06.631328 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-5a3e3d1c-c2db-46b1-92f4-502f1bfdbe6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:06.722180 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553a90c6-1847-4362-8822-ef624c24a817 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:06.723275 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1291/2579] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:06.723714 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-553a90c6-1847-4362-8822-ef624c24a817 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:06.725408 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212dae96-78c5-40ab-9ac8-dbc1ca23bcf2 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:06.725798 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212dae96-78c5-40ab-9ac8-dbc1ca23bcf2 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:06.726175 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212dae96-78c5-40ab-9ac8-dbc1ca23bcf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:06.744749 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553a90c6-1847-4362-8822-ef624c24a817 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:06.749591 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c7a2a-c2b4-4a4e-a6dc-82f6606de3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:06.750883 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c7a2a-c2b4-4a4e-a6dc-82f6606de3a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:06.751022 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c7a2a-c2b4-4a4e-a6dc-82f6606de3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:06.751232 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c7a2a-c2b4-4a4e-a6dc-82f6606de3a7 admin admin] PATH_INFO: `/v3/projects/3679cf44b5044fe5b483d3008f85e65c` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:06.753146 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0c7a2a-c2b4-4a4e-a6dc-82f6606de3a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3679cf44b5044fe5b483d3008f85e65c)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:06.756292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-212dae96-78c5-40ab-9ac8-dbc1ca23bcf2 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:06.756617 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0c7a2a-c2b4-4a4e-a6dc-82f6606de3a7 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:06.758108 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-212dae96-78c5-40ab-9ac8-dbc1ca23bcf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:06.769017 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1292/2580] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:06.773210 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212dae96-78c5-40ab-9ac8-dbc1ca23bcf2 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:06.786920 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212dae96-78c5-40ab-9ac8-dbc1ca23bcf2 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:06.793019 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a912304-93a0-435b-b38c-70ab1561cb46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:06.793972 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a912304-93a0-435b-b38c-70ab1561cb46 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:06.794655 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a912304-93a0-435b-b38c-70ab1561cb46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:06.794913 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a912304-93a0-435b-b38c-70ab1561cb46 admin admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:06.795873 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a912304-93a0-435b-b38c-70ab1561cb46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:28:06.796362 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a912304-93a0-435b-b38c-70ab1561cb46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:06.798065 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a912304-93a0-435b-b38c-70ab1561cb46 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:06.819187 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1293/2581] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:28:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:28:06.820963 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a912304-93a0-435b-b38c-70ab1561cb46 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:06.830456 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ff0c7a2a-c2b4-4a4e-a6dc-82f6606de3a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3679cf44b5044fe5b483d3008f85e65c', 'request_id': 'req-ff0c7a2a-c2b4-4a4e-a6dc-82f6606de3a7'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:06.834428 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a912304-93a0-435b-b38c-70ab1561cb46 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:06.837900 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ff0c7a2a-c2b4-4a4e-a6dc-82f6606de3a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3679cf44b5044fe5b483d3008f85e65c', 'request_id': 'req-ff0c7a2a-c2b4-4a4e-a6dc-82f6606de3a7'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:06.844330 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:06.848251 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ff0c7a2a-c2b4-4a4e-a6dc-82f6606de3a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3679cf44b5044fe5b483d3008f85e65c', 'request_id': 'req-ff0c7a2a-c2b4-4a4e-a6dc-82f6606de3a7'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:06.860145 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ff0c7a2a-c2b4-4a4e-a6dc-82f6606de3a7 admin admin] The token cache is being invalidate because project 3679cf44b5044fe5b483d3008f85e65c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:28:06.860434 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ff0c7a2a-c2b4-4a4e-a6dc-82f6606de3a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:06.861096 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-ff0c7a2a-c2b4-4a4e-a6dc-82f6606de3a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:06.862947 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1289/2582] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:28:06 2022] DELETE /identity/v3/projects/3679cf44b5044fe5b483d3008f85e65c => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:06.866699 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c7a2a-c2b4-4a4e-a6dc-82f6606de3a7 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:06.869527 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ff0c7a2a-c2b4-4a4e-a6dc-82f6606de3a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:06.902517 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521817f1-7f9c-44b8-b1b4-e3230b3d619d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:06.903673 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521817f1-7f9c-44b8-b1b4-e3230b3d619d admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:06.903963 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521817f1-7f9c-44b8-b1b4-e3230b3d619d admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:06.904242 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521817f1-7f9c-44b8-b1b4-e3230b3d619d admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:06.904938 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521817f1-7f9c-44b8-b1b4-e3230b3d619d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:06.906906 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521817f1-7f9c-44b8-b1b4-e3230b3d619d admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:06.940475 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1294/2583] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:28:06 2022] POST /identity/v3/projects => generated 371 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:28:06.943221 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c7a2a-c2b4-4a4e-a6dc-82f6606de3a7 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:06.944353 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521817f1-7f9c-44b8-b1b4-e3230b3d619d admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:06.946003 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-521817f1-7f9c-44b8-b1b4-e3230b3d619d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:07.016710 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521817f1-7f9c-44b8-b1b4-e3230b3d619d admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:07.021945 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f0ee0-0366-4bd1-96e7-606e0f9e7587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:07.022825 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f0ee0-0366-4bd1-96e7-606e0f9e7587 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:07.023113 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f0ee0-0366-4bd1-96e7-606e0f9e7587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:07.023474 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f0ee0-0366-4bd1-96e7-606e0f9e7587 admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:07.024204 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f0ee0-0366-4bd1-96e7-606e0f9e7587 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:07.025207 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb139bc-01f3-4e97-baec-042dd3653029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:07.026260 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb139bc-01f3-4e97-baec-042dd3653029 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:07.026380 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f0ee0-0366-4bd1-96e7-606e0f9e7587 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:07.026705 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb139bc-01f3-4e97-baec-042dd3653029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:07.027307 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb139bc-01f3-4e97-baec-042dd3653029 admin admin] PATH_INFO: `/v3/users/fc5aa39fb1144552a519b20596845924` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:07.029676 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb139bc-01f3-4e97-baec-042dd3653029 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc5aa39fb1144552a519b20596845924)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:07.031947 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb139bc-01f3-4e97-baec-042dd3653029 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:07.052596 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1295/2584] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:28:06 2022] POST /identity/v3/users => generated 286 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:28:07.056754 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f0ee0-0366-4bd1-96e7-606e0f9e7587 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:07.070466 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f0ee0-0366-4bd1-96e7-606e0f9e7587 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:07.080525 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc1be4a-db5f-40dc-bd94-4289c8df17fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:07.081224 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1be4a-db5f-40dc-bd94-4289c8df17fc admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:07.081567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1be4a-db5f-40dc-bd94-4289c8df17fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:07.081854 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1be4a-db5f-40dc-bd94-4289c8df17fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:07.082803 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc1be4a-db5f-40dc-bd94-4289c8df17fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:07.084578 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc1be4a-db5f-40dc-bd94-4289c8df17fc admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:07.106680 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:07.108045 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1296/2585] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:28:07.109387 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc1be4a-db5f-40dc-bd94-4289c8df17fc admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:07.124420 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc1be4a-db5f-40dc-bd94-4289c8df17fc admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:07.134857 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721af787-f872-426b-a381-96df0ec4cef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:07.136013 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721af787-f872-426b-a381-96df0ec4cef9 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:07.136444 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721af787-f872-426b-a381-96df0ec4cef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:07.136895 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721af787-f872-426b-a381-96df0ec4cef9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:07.138352 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721af787-f872-426b-a381-96df0ec4cef9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:07.140174 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721af787-f872-426b-a381-96df0ec4cef9 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:07.140324 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-edb139bc-01f3-4e97-baec-042dd3653029 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc5aa39fb1144552a519b20596845924'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:07.140916 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-edb139bc-01f3-4e97-baec-042dd3653029 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc5aa39fb1144552a519b20596845924'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:07.170312 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-edb139bc-01f3-4e97-baec-042dd3653029 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc5aa39fb1144552a519b20596845924'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:07.185841 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1297/2586] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:07 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:28:07.187189 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721af787-f872-426b-a381-96df0ec4cef9 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:07.188473 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-edb139bc-01f3-4e97-baec-042dd3653029 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc5aa39fb1144552a519b20596845924'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:07.189656 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-721af787-f872-426b-a381-96df0ec4cef9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:07.224575 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-edb139bc-01f3-4e97-baec-042dd3653029 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc5aa39fb1144552a519b20596845924'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:07.226399 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721af787-f872-426b-a381-96df0ec4cef9 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:07.233903 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1290/2587] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:28:06 2022] DELETE /identity/v3/users/fc5aa39fb1144552a519b20596845924 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:28:07.260624 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4efe170-5092-4c62-ba0e-6b611bc388e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:07.264805 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4efe170-5092-4c62-ba0e-6b611bc388e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:07.265095 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4efe170-5092-4c62-ba0e-6b611bc388e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:07.265384 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4efe170-5092-4c62-ba0e-6b611bc388e5 admin admin] PATH_INFO: `/v3/projects/4290e932e2484d2999ae35286e04407a/users/4a41274fce70413a82632dcbe9ddb28a/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:07.293835 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4efe170-5092-4c62-ba0e-6b611bc388e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4290e932e2484d2999ae35286e04407a, user_id=4a41274fce70413a82632dcbe9ddb28a, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:07.297816 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4efe170-5092-4c62-ba0e-6b611bc388e5 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:07.327916 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1298/2588] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:28:07 2022] PUT /identity/v3/projects/4290e932e2484d2999ae35286e04407a/users/4a41274fce70413a82632dcbe9ddb28a/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:07.335069 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596639e-6eb9-487c-8bd6-77dd17433deb None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:07.335819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596639e-6eb9-487c-8bd6-77dd17433deb None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:07.335819 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596639e-6eb9-487c-8bd6-77dd17433deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:07.370756 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-0596639e-6eb9-487c-8bd6-77dd17433deb None None] MFA Rules not processed for user `4a41274fce70413a82632dcbe9ddb28a`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:07.372679 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0596639e-6eb9-487c-8bd6-77dd17433deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:07.471360 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1291/2589] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:07.478655 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4efe170-5092-4c62-ba0e-6b611bc388e5 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:07.480907 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-c4efe170-5092-4c62-ba0e-6b611bc388e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:07.613599 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:07.738477 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4efe170-5092-4c62-ba0e-6b611bc388e5 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:07.760923 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b37d5c3-a2c2-4464-b680-a8fa577fbcd5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:07.763450 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b37d5c3-a2c2-4464-b680-a8fa577fbcd5 service nova] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:07.764418 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b37d5c3-a2c2-4464-b680-a8fa577fbcd5 service nova] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:07.765857 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b37d5c3-a2c2-4464-b680-a8fa577fbcd5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:07.796621 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b37d5c3-a2c2-4464-b680-a8fa577fbcd5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:07.799635 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b37d5c3-a2c2-4464-b680-a8fa577fbcd5 service nova] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:07.829045 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1299/2590] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:28:07.947586 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0596639e-6eb9-487c-8bd6-77dd17433deb None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:07.961068 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0596639e-6eb9-487c-8bd6-77dd17433deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:08.153552 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0596639e-6eb9-487c-8bd6-77dd17433deb None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:08.163347 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113a1d18-91e1-4e00-a962-82c7a223c87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:08.164684 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a1d18-91e1-4e00-a962-82c7a223c87e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:08.165151 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a1d18-91e1-4e00-a962-82c7a223c87e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:08.165571 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a1d18-91e1-4e00-a962-82c7a223c87e admin admin] PATH_INFO: `/v3/projects/84a50f629d344241be76e3d5a11af9f6` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:08.167576 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113a1d18-91e1-4e00-a962-82c7a223c87e admin admin] RBAC: Authorizing `identity:delete_project(project_id=84a50f629d344241be76e3d5a11af9f6)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:08.170309 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113a1d18-91e1-4e00-a962-82c7a223c87e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:08.253030 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:08.269513 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-113a1d18-91e1-4e00-a962-82c7a223c87e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84a50f629d344241be76e3d5a11af9f6', 'request_id': 'req-113a1d18-91e1-4e00-a962-82c7a223c87e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:08.283413 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-113a1d18-91e1-4e00-a962-82c7a223c87e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84a50f629d344241be76e3d5a11af9f6', 'request_id': 'req-113a1d18-91e1-4e00-a962-82c7a223c87e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:08.302220 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-113a1d18-91e1-4e00-a962-82c7a223c87e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84a50f629d344241be76e3d5a11af9f6', 'request_id': 'req-113a1d18-91e1-4e00-a962-82c7a223c87e'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:08.318242 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-113a1d18-91e1-4e00-a962-82c7a223c87e admin admin] The token cache is being invalidate because project 84a50f629d344241be76e3d5a11af9f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:28:08.318242 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-113a1d18-91e1-4e00-a962-82c7a223c87e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:08.321835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-113a1d18-91e1-4e00-a962-82c7a223c87e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:08.328581 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1292/2591] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:28:07 2022] DELETE /identity/v3/projects/84a50f629d344241be76e3d5a11af9f6 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:08.330151 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e20345-c241-4f91-b331-acb1ed87c12e None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:08.330738 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e20345-c241-4f91-b331-acb1ed87c12e None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:08.330738 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e20345-c241-4f91-b331-acb1ed87c12e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:08.363854 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-32e20345-c241-4f91-b331-acb1ed87c12e None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:08.365542 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-32e20345-c241-4f91-b331-acb1ed87c12e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:08.530546 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1300/2592] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:08.541281 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06040d48-a7a7-488f-beef-eb6a8573cb73 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:08.541632 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06040d48-a7a7-488f-beef-eb6a8573cb73 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:08.541919 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06040d48-a7a7-488f-beef-eb6a8573cb73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:08.578974 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-06040d48-a7a7-488f-beef-eb6a8573cb73 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:08.580588 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-06040d48-a7a7-488f-beef-eb6a8573cb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:08.596605 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e20345-c241-4f91-b331-acb1ed87c12e None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:08.607835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1293/2593] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:08.622566 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e20345-c241-4f91-b331-acb1ed87c12e None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:08.631323 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ca9c4-c9a6-4b0b-ae80-f0b50105ed2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:08.632095 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ca9c4-c9a6-4b0b-ae80-f0b50105ed2f admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:08.632444 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ca9c4-c9a6-4b0b-ae80-f0b50105ed2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:08.632790 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ca9c4-c9a6-4b0b-ae80-f0b50105ed2f admin admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:08.639815 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47ca9c4-c9a6-4b0b-ae80-f0b50105ed2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:28:08.640159 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47ca9c4-c9a6-4b0b-ae80-f0b50105ed2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:08.642151 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47ca9c4-c9a6-4b0b-ae80-f0b50105ed2f admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:08.660339 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1301/2594] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:28:08.661090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06040d48-a7a7-488f-beef-eb6a8573cb73 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:08.675048 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06040d48-a7a7-488f-beef-eb6a8573cb73 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:08.689628 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df15a8-9211-45e9-95b1-277420e750b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:08.690300 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df15a8-9211-45e9-95b1-277420e750b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:08.690576 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df15a8-9211-45e9-95b1-277420e750b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:08.690848 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df15a8-9211-45e9-95b1-277420e750b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:08.691572 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59df15a8-9211-45e9-95b1-277420e750b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:08.693735 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59df15a8-9211-45e9-95b1-277420e750b0 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:08.735591 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1294/2595] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:28:08 2022] POST /identity/v3/projects => generated 373 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:28:08.737938 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ca9c4-c9a6-4b0b-ae80-f0b50105ed2f admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:08.749488 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ca9c4-c9a6-4b0b-ae80-f0b50105ed2f admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:08.820883 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac6a74-aa29-466f-9aea-d67c2cfc8650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:08.821878 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac6a74-aa29-466f-9aea-d67c2cfc8650 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:08.822164 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac6a74-aa29-466f-9aea-d67c2cfc8650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:08.822437 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac6a74-aa29-466f-9aea-d67c2cfc8650 admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:08.823145 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ac6a74-aa29-466f-9aea-d67c2cfc8650 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:08.825808 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ac6a74-aa29-466f-9aea-d67c2cfc8650 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:08.854350 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df15a8-9211-45e9-95b1-277420e750b0 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:08.854774 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1302/2596] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:28:08 2022] POST /identity/v3/users => generated 287 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:28:08.865577 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df15a8-9211-45e9-95b1-277420e750b0 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:08.870464 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e01859-9af2-43b2-af7b-0a3006c50870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:08.871242 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e01859-9af2-43b2-af7b-0a3006c50870 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:08.871402 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e01859-9af2-43b2-af7b-0a3006c50870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:08.872242 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e01859-9af2-43b2-af7b-0a3006c50870 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:08.873039 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e01859-9af2-43b2-af7b-0a3006c50870 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:08.874689 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e01859-9af2-43b2-af7b-0a3006c50870 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:08.888428 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1295/2597] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:28:08.892479 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac6a74-aa29-466f-9aea-d67c2cfc8650 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:08.902528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac6a74-aa29-466f-9aea-d67c2cfc8650 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:08.907366 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa56fa-0e32-476f-9081-297cd9610207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:08.908066 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa56fa-0e32-476f-9081-297cd9610207 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:08.908340 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa56fa-0e32-476f-9081-297cd9610207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:08.908617 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa56fa-0e32-476f-9081-297cd9610207 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:08.909359 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fa56fa-0e32-476f-9081-297cd9610207 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:08.910957 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fa56fa-0e32-476f-9081-297cd9610207 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:08.922339 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1303/2598] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:28:08.926466 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e01859-9af2-43b2-af7b-0a3006c50870 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:08.937787 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e01859-9af2-43b2-af7b-0a3006c50870 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:08.942503 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31443c87-f436-4433-ae99-54b4ab2b87bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:08.943415 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31443c87-f436-4433-ae99-54b4ab2b87bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:08.943733 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31443c87-f436-4433-ae99-54b4ab2b87bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:08.944040 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31443c87-f436-4433-ae99-54b4ab2b87bd admin admin] PATH_INFO: `/v3/projects/0ab5968c3bcb4b7091a5f98fe282323c/users/0927c0ae48a1478487058ca1d05ae5c9/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:08.975791 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31443c87-f436-4433-ae99-54b4ab2b87bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ab5968c3bcb4b7091a5f98fe282323c, user_id=0927c0ae48a1478487058ca1d05ae5c9, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:08.979392 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31443c87-f436-4433-ae99-54b4ab2b87bd admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:09.016844 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a9c7e-a434-4c02-9e59-8e81981560a2 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:09.017141 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a9c7e-a434-4c02-9e59-8e81981560a2 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:09.017423 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a9c7e-a434-4c02-9e59-8e81981560a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:09.017730 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1296/2599] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:28:08 2022] PUT /identity/v3/projects/0ab5968c3bcb4b7091a5f98fe282323c/users/0927c0ae48a1478487058ca1d05ae5c9/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:28:09.065634 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-102a9c7e-a434-4c02-9e59-8e81981560a2 None None] MFA Rules not processed for user `0927c0ae48a1478487058ca1d05ae5c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:09.068289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-102a9c7e-a434-4c02-9e59-8e81981560a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:09.096851 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:09.104327 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31443c87-f436-4433-ae99-54b4ab2b87bd admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:09.108881 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-31443c87-f436-4433-ae99-54b4ab2b87bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:09.223603 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1304/2600] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:09.235805 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a9c7e-a434-4c02-9e59-8e81981560a2 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:09.238574 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-102a9c7e-a434-4c02-9e59-8e81981560a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:09.318248 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31443c87-f436-4433-ae99-54b4ab2b87bd admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:09.325953 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad1b2a3-add3-4293-860c-7ba10887497c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:09.326629 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1b2a3-add3-4293-860c-7ba10887497c service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:09.326914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1b2a3-add3-4293-860c-7ba10887497c service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:09.327240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1b2a3-add3-4293-860c-7ba10887497c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:09.329324 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-aad1b2a3-add3-4293-860c-7ba10887497c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:09.345275 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad1b2a3-add3-4293-860c-7ba10887497c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:09.364016 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad1b2a3-add3-4293-860c-7ba10887497c service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:09.380847 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1297/2601] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:28:09.382256 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a9c7e-a434-4c02-9e59-8e81981560a2 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:09.387691 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0d220-0e2b-4e51-b177-84a69bf11e61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:09.388448 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0d220-0e2b-4e51-b177-84a69bf11e61 service cinder] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:09.388801 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0d220-0e2b-4e51-b177-84a69bf11e61 service cinder] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:09.389269 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0d220-0e2b-4e51-b177-84a69bf11e61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:09.399710 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e0d220-0e2b-4e51-b177-84a69bf11e61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:09.401489 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e0d220-0e2b-4e51-b177-84a69bf11e61 service cinder] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:09.415370 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1305/2602] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:28:11.000092 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad1b2a3-add3-4293-860c-7ba10887497c service nova] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:11.000092 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-aad1b2a3-add3-4293-860c-7ba10887497c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:11.074130 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:11.104878 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad1b2a3-add3-4293-860c-7ba10887497c service nova] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:11.113178 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fa021f-5a07-4a07-a929-0917723e98ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:11.114127 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa021f-5a07-4a07-a929-0917723e98ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:11.114411 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa021f-5a07-4a07-a929-0917723e98ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:11.114686 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fa021f-5a07-4a07-a929-0917723e98ce admin admin] PATH_INFO: `/v3/users/4a41274fce70413a82632dcbe9ddb28a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:11.115935 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fa021f-5a07-4a07-a929-0917723e98ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a41274fce70413a82632dcbe9ddb28a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:11.117929 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fa021f-5a07-4a07-a929-0917723e98ce admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:11.194947 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-89fa021f-5a07-4a07-a929-0917723e98ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a41274fce70413a82632dcbe9ddb28a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:11.196316 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-89fa021f-5a07-4a07-a929-0917723e98ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a41274fce70413a82632dcbe9ddb28a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:11.215269 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-89fa021f-5a07-4a07-a929-0917723e98ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a41274fce70413a82632dcbe9ddb28a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:11.225743 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-89fa021f-5a07-4a07-a929-0917723e98ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a41274fce70413a82632dcbe9ddb28a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:11.234383 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-89fa021f-5a07-4a07-a929-0917723e98ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a41274fce70413a82632dcbe9ddb28a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:11.240923 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1298/2603] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:28:10 2022] DELETE /identity/v3/users/4a41274fce70413a82632dcbe9ddb28a => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:11.245887 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d5581-8fb7-421f-baab-89dd031aeb61 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:11.246459 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d5581-8fb7-421f-baab-89dd031aeb61 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:11.246614 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d5581-8fb7-421f-baab-89dd031aeb61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:11.268122 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-f84d5581-8fb7-421f-baab-89dd031aeb61 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:11.269517 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-f84d5581-8fb7-421f-baab-89dd031aeb61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:11.339512 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1306/2604] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:11.350903 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fa021f-5a07-4a07-a929-0917723e98ce admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:11.357896 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-89fa021f-5a07-4a07-a929-0917723e98ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:11.432675 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fa021f-5a07-4a07-a929-0917723e98ce admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:11.438686 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d933d-1e29-4ebc-a109-3f28436670b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:11.439645 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d933d-1e29-4ebc-a109-3f28436670b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:11.439933 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d933d-1e29-4ebc-a109-3f28436670b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:11.440231 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d933d-1e29-4ebc-a109-3f28436670b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:11.451949 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193d933d-1e29-4ebc-a109-3f28436670b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:11.453585 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193d933d-1e29-4ebc-a109-3f28436670b0 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:11.465054 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1299/2605] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:28:11.774750 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d5581-8fb7-421f-baab-89dd031aeb61 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:11.776418 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-f84d5581-8fb7-421f-baab-89dd031aeb61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:11.790454 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d5581-8fb7-421f-baab-89dd031aeb61 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:11.796911 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f4615f-9869-4e7b-a0fd-d4d51844fad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:11.797978 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f4615f-9869-4e7b-a0fd-d4d51844fad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:11.798365 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f4615f-9869-4e7b-a0fd-d4d51844fad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:11.798798 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f4615f-9869-4e7b-a0fd-d4d51844fad8 admin admin] PATH_INFO: `/v3/projects/4290e932e2484d2999ae35286e04407a` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:11.800169 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f4615f-9869-4e7b-a0fd-d4d51844fad8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4290e932e2484d2999ae35286e04407a)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:11.802371 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f4615f-9869-4e7b-a0fd-d4d51844fad8 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:11.854645 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-91f4615f-9869-4e7b-a0fd-d4d51844fad8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4290e932e2484d2999ae35286e04407a', 'request_id': 'req-91f4615f-9869-4e7b-a0fd-d4d51844fad8'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:11.860938 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-91f4615f-9869-4e7b-a0fd-d4d51844fad8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4290e932e2484d2999ae35286e04407a', 'request_id': 'req-91f4615f-9869-4e7b-a0fd-d4d51844fad8'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:11.865991 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:11.869920 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-91f4615f-9869-4e7b-a0fd-d4d51844fad8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4290e932e2484d2999ae35286e04407a', 'request_id': 'req-91f4615f-9869-4e7b-a0fd-d4d51844fad8'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:11.883260 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-91f4615f-9869-4e7b-a0fd-d4d51844fad8 admin admin] The token cache is being invalidate because project 4290e932e2484d2999ae35286e04407a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:28:11.883576 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-91f4615f-9869-4e7b-a0fd-d4d51844fad8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:11.884231 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-91f4615f-9869-4e7b-a0fd-d4d51844fad8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:11.885866 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1307/2606] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:28:11 2022] DELETE /identity/v3/projects/4290e932e2484d2999ae35286e04407a => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:11.891348 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792b2ff-40be-428d-b2ff-366f2f3121d3 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:11.891678 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792b2ff-40be-428d-b2ff-366f2f3121d3 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:11.891961 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792b2ff-40be-428d-b2ff-366f2f3121d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:11.909643 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-8792b2ff-40be-428d-b2ff-366f2f3121d3 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:11.910972 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8792b2ff-40be-428d-b2ff-366f2f3121d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:11.977575 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1300/2607] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:11.985511 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca640d31-7cfd-4250-a5b0-b36932f9e0d3 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:11.985798 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca640d31-7cfd-4250-a5b0-b36932f9e0d3 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:11.986075 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca640d31-7cfd-4250-a5b0-b36932f9e0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:12.011998 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-ca640d31-7cfd-4250-a5b0-b36932f9e0d3 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:12.013276 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-ca640d31-7cfd-4250-a5b0-b36932f9e0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:12.021607 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1308/2608] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:12.025637 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792b2ff-40be-428d-b2ff-366f2f3121d3 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:12.035839 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792b2ff-40be-428d-b2ff-366f2f3121d3 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:12.040470 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a396f960-193c-4290-9c40-95af1acb8e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:12.041182 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396f960-193c-4290-9c40-95af1acb8e43 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:12.041468 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396f960-193c-4290-9c40-95af1acb8e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:12.041741 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396f960-193c-4290-9c40-95af1acb8e43 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:12.042632 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a396f960-193c-4290-9c40-95af1acb8e43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:28:12.042844 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a396f960-193c-4290-9c40-95af1acb8e43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:12.047365 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a396f960-193c-4290-9c40-95af1acb8e43 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:12.058780 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:12.070815 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1301/2609] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:28:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:28:12.071393 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca640d31-7cfd-4250-a5b0-b36932f9e0d3 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:12.081294 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca640d31-7cfd-4250-a5b0-b36932f9e0d3 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:12.092813 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6efbaf-8c52-4c96-8a12-589fb6a365d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:12.092813 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6efbaf-8c52-4c96-8a12-589fb6a365d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:12.092813 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6efbaf-8c52-4c96-8a12-589fb6a365d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:12.092813 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6efbaf-8c52-4c96-8a12-589fb6a365d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:12.095230 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6efbaf-8c52-4c96-8a12-589fb6a365d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:12.095230 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6efbaf-8c52-4c96-8a12-589fb6a365d2 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:12.120637 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1309/2610] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:28:12 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:28:12.132407 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a396f960-193c-4290-9c40-95af1acb8e43 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:12.142892 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a396f960-193c-4290-9c40-95af1acb8e43 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:12.221130 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bee745-0a00-4490-ad60-455a2675e73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:12.222576 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bee745-0a00-4490-ad60-455a2675e73e admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:12.223451 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bee745-0a00-4490-ad60-455a2675e73e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:12.223964 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bee745-0a00-4490-ad60-455a2675e73e admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:12.225110 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bee745-0a00-4490-ad60-455a2675e73e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:12.227346 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bee745-0a00-4490-ad60-455a2675e73e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:12.262457 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1302/2611] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:28:12 2022] POST /identity/v3/users => generated 284 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:28:12.266871 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6efbaf-8c52-4c96-8a12-589fb6a365d2 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:12.281554 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6efbaf-8c52-4c96-8a12-589fb6a365d2 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:12.287320 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b36b6-1d1f-4f37-b0ce-9f6a1e47e143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:12.288159 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b36b6-1d1f-4f37-b0ce-9f6a1e47e143 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:12.288507 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b36b6-1d1f-4f37-b0ce-9f6a1e47e143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:12.288851 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b36b6-1d1f-4f37-b0ce-9f6a1e47e143 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:12.289667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261b36b6-1d1f-4f37-b0ce-9f6a1e47e143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:12.291351 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261b36b6-1d1f-4f37-b0ce-9f6a1e47e143 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:12.303623 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1310/2612] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:28:12.308052 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bee745-0a00-4490-ad60-455a2675e73e admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:12.320189 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bee745-0a00-4490-ad60-455a2675e73e admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:12.326153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9658839-e5a0-4683-94a6-27f44d263801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:12.326836 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9658839-e5a0-4683-94a6-27f44d263801 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:12.327115 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9658839-e5a0-4683-94a6-27f44d263801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:12.327498 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9658839-e5a0-4683-94a6-27f44d263801 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:12.328144 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9658839-e5a0-4683-94a6-27f44d263801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:12.329747 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9658839-e5a0-4683-94a6-27f44d263801 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:12.340983 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1303/2613] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:28:12.345301 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b36b6-1d1f-4f37-b0ce-9f6a1e47e143 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:12.358932 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b36b6-1d1f-4f37-b0ce-9f6a1e47e143 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:12.372553 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d740f-4e89-412b-9c82-2ab5ca3b65a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:12.373111 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d740f-4e89-412b-9c82-2ab5ca3b65a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:12.373392 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d740f-4e89-412b-9c82-2ab5ca3b65a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:12.373667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d740f-4e89-412b-9c82-2ab5ca3b65a6 admin admin] PATH_INFO: `/v3/projects/705fba370d7a42b18e24d3772bcccc0e/users/e708995dd77c4e5cb9430286bf5991a9/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:12.388796 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70d740f-4e89-412b-9c82-2ab5ca3b65a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=705fba370d7a42b18e24d3772bcccc0e, user_id=e708995dd77c4e5cb9430286bf5991a9, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:12.391508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70d740f-4e89-412b-9c82-2ab5ca3b65a6 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:12.407846 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1311/2614] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:28:12 2022] PUT /identity/v3/projects/705fba370d7a42b18e24d3772bcccc0e/users/e708995dd77c4e5cb9430286bf5991a9/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:12.421299 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18c071-7d9b-4ccf-99a6-f1ec532dc011 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:12.421765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18c071-7d9b-4ccf-99a6-f1ec532dc011 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:12.422059 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18c071-7d9b-4ccf-99a6-f1ec532dc011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:12.452956 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-1d18c071-7d9b-4ccf-99a6-f1ec532dc011 None None] MFA Rules not processed for user `e708995dd77c4e5cb9430286bf5991a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:12.454384 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1d18c071-7d9b-4ccf-99a6-f1ec532dc011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:12.522611 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1304/2615] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:12 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:12.534068 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d740f-4e89-412b-9c82-2ab5ca3b65a6 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:12.536091 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-d70d740f-4e89-412b-9c82-2ab5ca3b65a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:12.596515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d740f-4e89-412b-9c82-2ab5ca3b65a6 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:12.600997 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d75e0e-15e9-463b-8d10-82bfe78a5cbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:12.601642 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d75e0e-15e9-463b-8d10-82bfe78a5cbc service cinder] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:12.602173 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d75e0e-15e9-463b-8d10-82bfe78a5cbc service cinder] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:12.602173 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d75e0e-15e9-463b-8d10-82bfe78a5cbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:12.612342 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d75e0e-15e9-463b-8d10-82bfe78a5cbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:12.613856 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d75e0e-15e9-463b-8d10-82bfe78a5cbc service cinder] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:12.623393 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1312/2616] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:12 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:28:12.688806 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d18c071-7d9b-4ccf-99a6-f1ec532dc011 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:12.690249 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1d18c071-7d9b-4ccf-99a6-f1ec532dc011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:12.758196 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d18c071-7d9b-4ccf-99a6-f1ec532dc011 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:12.763674 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485cc39a-b991-4f44-8e64-424dc22bf0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:12.764585 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485cc39a-b991-4f44-8e64-424dc22bf0d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:12.764838 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485cc39a-b991-4f44-8e64-424dc22bf0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:12.765097 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485cc39a-b991-4f44-8e64-424dc22bf0d4 admin admin] PATH_INFO: `/v3/users/0927c0ae48a1478487058ca1d05ae5c9` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:12.766303 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485cc39a-b991-4f44-8e64-424dc22bf0d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0927c0ae48a1478487058ca1d05ae5c9)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:12.768261 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485cc39a-b991-4f44-8e64-424dc22bf0d4 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:12.774383 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:12.819705 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-485cc39a-b991-4f44-8e64-424dc22bf0d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0927c0ae48a1478487058ca1d05ae5c9'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:12.820498 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-485cc39a-b991-4f44-8e64-424dc22bf0d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0927c0ae48a1478487058ca1d05ae5c9'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:12.833331 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-485cc39a-b991-4f44-8e64-424dc22bf0d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0927c0ae48a1478487058ca1d05ae5c9'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:12.842461 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-485cc39a-b991-4f44-8e64-424dc22bf0d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0927c0ae48a1478487058ca1d05ae5c9'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:12.849133 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-485cc39a-b991-4f44-8e64-424dc22bf0d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0927c0ae48a1478487058ca1d05ae5c9'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:12.856271 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1305/2617] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:28:12 2022] DELETE /identity/v3/users/0927c0ae48a1478487058ca1d05ae5c9 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:12.860945 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae15cbe2-bb39-4b10-9101-d16e61ac3bf1 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:12.861432 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae15cbe2-bb39-4b10-9101-d16e61ac3bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:12.861552 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae15cbe2-bb39-4b10-9101-d16e61ac3bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:12.885068 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-ae15cbe2-bb39-4b10-9101-d16e61ac3bf1 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:12.887790 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-ae15cbe2-bb39-4b10-9101-d16e61ac3bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:12.964489 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1313/2618] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:12.974836 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485cc39a-b991-4f44-8e64-424dc22bf0d4 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:12.976377 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-485cc39a-b991-4f44-8e64-424dc22bf0d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:13.035327 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485cc39a-b991-4f44-8e64-424dc22bf0d4 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:13.039042 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c496f-25ec-4532-b09e-115d51f476e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:13.039714 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c496f-25ec-4532-b09e-115d51f476e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:13.039993 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c496f-25ec-4532-b09e-115d51f476e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:13.040281 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c496f-25ec-4532-b09e-115d51f476e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:13.049773 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825c496f-25ec-4532-b09e-115d51f476e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:13.051282 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825c496f-25ec-4532-b09e-115d51f476e9 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:13.061244 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1306/2619] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:28:13.343896 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae15cbe2-bb39-4b10-9101-d16e61ac3bf1 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:13.345950 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-ae15cbe2-bb39-4b10-9101-d16e61ac3bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:13.368741 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae15cbe2-bb39-4b10-9101-d16e61ac3bf1 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:13.368887 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c496f-25ec-4532-b09e-115d51f476e9 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:13.374756 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47939518-3d60-4c6d-b88b-5b7971b0c18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:13.378514 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c496f-25ec-4532-b09e-115d51f476e9 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:13.382148 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5332da49-af5b-4af4-bc90-9911cc285a43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:13.382767 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47939518-3d60-4c6d-b88b-5b7971b0c18d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:13.382871 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332da49-af5b-4af4-bc90-9911cc285a43 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:13.383047 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332da49-af5b-4af4-bc90-9911cc285a43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:13.383142 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47939518-3d60-4c6d-b88b-5b7971b0c18d admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:13.383319 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332da49-af5b-4af4-bc90-9911cc285a43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:13.383418 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47939518-3d60-4c6d-b88b-5b7971b0c18d admin admin] PATH_INFO: `/v3/projects/0ab5968c3bcb4b7091a5f98fe282323c` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:13.384629 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47939518-3d60-4c6d-b88b-5b7971b0c18d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ab5968c3bcb4b7091a5f98fe282323c)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:13.385111 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5332da49-af5b-4af4-bc90-9911cc285a43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:13.386989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47939518-3d60-4c6d-b88b-5b7971b0c18d admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:13.398974 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5332da49-af5b-4af4-bc90-9911cc285a43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:13.400453 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5332da49-af5b-4af4-bc90-9911cc285a43 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:13.410572 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1307/2620] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:28:13.412130 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:13.440144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-47939518-3d60-4c6d-b88b-5b7971b0c18d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ab5968c3bcb4b7091a5f98fe282323c', 'request_id': 'req-47939518-3d60-4c6d-b88b-5b7971b0c18d'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:13.446723 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-47939518-3d60-4c6d-b88b-5b7971b0c18d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ab5968c3bcb4b7091a5f98fe282323c', 'request_id': 'req-47939518-3d60-4c6d-b88b-5b7971b0c18d'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:13.454565 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-47939518-3d60-4c6d-b88b-5b7971b0c18d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ab5968c3bcb4b7091a5f98fe282323c', 'request_id': 'req-47939518-3d60-4c6d-b88b-5b7971b0c18d'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:13.467333 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-47939518-3d60-4c6d-b88b-5b7971b0c18d admin admin] The token cache is being invalidate because project 0ab5968c3bcb4b7091a5f98fe282323c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:28:13.467659 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-47939518-3d60-4c6d-b88b-5b7971b0c18d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:13.468327 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-47939518-3d60-4c6d-b88b-5b7971b0c18d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:13.471710 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1314/2621] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:28:13 2022] DELETE /identity/v3/projects/0ab5968c3bcb4b7091a5f98fe282323c => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:13.475094 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a1217-023f-43ea-b128-22d6dab274e6 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:13.475385 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a1217-023f-43ea-b128-22d6dab274e6 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:13.475699 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a1217-023f-43ea-b128-22d6dab274e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:13.495375 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-990a1217-023f-43ea-b128-22d6dab274e6 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:13.497320 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-990a1217-023f-43ea-b128-22d6dab274e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:13.597765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1308/2622] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:13.606882 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc551700-6be2-415e-becf-c93239b33678 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:13.607186 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc551700-6be2-415e-becf-c93239b33678 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:13.607450 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc551700-6be2-415e-becf-c93239b33678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:13.633404 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-cc551700-6be2-415e-becf-c93239b33678 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:13.634916 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-cc551700-6be2-415e-becf-c93239b33678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:13.648943 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1315/2623] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:13.650742 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a1217-023f-43ea-b128-22d6dab274e6 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:13.663418 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a1217-023f-43ea-b128-22d6dab274e6 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:13.670129 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b31c98d-e45b-4ffc-adbb-8eeb9aed2d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:13.671080 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b31c98d-e45b-4ffc-adbb-8eeb9aed2d5c admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:13.671362 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b31c98d-e45b-4ffc-adbb-8eeb9aed2d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:13.671721 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b31c98d-e45b-4ffc-adbb-8eeb9aed2d5c admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:13.672596 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b31c98d-e45b-4ffc-adbb-8eeb9aed2d5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:28:13.672928 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b31c98d-e45b-4ffc-adbb-8eeb9aed2d5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:13.675292 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b31c98d-e45b-4ffc-adbb-8eeb9aed2d5c admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:13.702823 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1309/2624] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:28:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:28:13.707082 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc551700-6be2-415e-becf-c93239b33678 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:13.726080 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc551700-6be2-415e-becf-c93239b33678 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:13.748130 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02853b43-3f78-414f-9215-6e4d9c3c3491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:13.748820 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02853b43-3f78-414f-9215-6e4d9c3c3491 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:13.749108 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02853b43-3f78-414f-9215-6e4d9c3c3491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:13.749384 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02853b43-3f78-414f-9215-6e4d9c3c3491 admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:13.750096 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02853b43-3f78-414f-9215-6e4d9c3c3491 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:13.754124 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02853b43-3f78-414f-9215-6e4d9c3c3491 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:13.791572 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1316/2625] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:28:13 2022] POST /identity/v3/projects => generated 397 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:28:13.795110 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b31c98d-e45b-4ffc-adbb-8eeb9aed2d5c admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:13.800537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:13.807615 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b31c98d-e45b-4ffc-adbb-8eeb9aed2d5c admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:13.808776 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02853b43-3f78-414f-9215-6e4d9c3c3491 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:13.811274 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-02853b43-3f78-414f-9215-6e4d9c3c3491 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:13.880859 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e9356c-725a-4d7a-91c1-659e44522e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:13.882331 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e9356c-725a-4d7a-91c1-659e44522e16 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:13.882696 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e9356c-725a-4d7a-91c1-659e44522e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:13.883737 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e9356c-725a-4d7a-91c1-659e44522e16 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:13.884443 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e9356c-725a-4d7a-91c1-659e44522e16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:13.885881 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02853b43-3f78-414f-9215-6e4d9c3c3491 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:13.886702 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e9356c-725a-4d7a-91c1-659e44522e16 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:13.889827 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e5882-6db6-4c30-aba9-a0685bbb57e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:13.890483 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e5882-6db6-4c30-aba9-a0685bbb57e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:13.890824 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e5882-6db6-4c30-aba9-a0685bbb57e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:13.891114 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e5882-6db6-4c30-aba9-a0685bbb57e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:13.892946 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-437e5882-6db6-4c30-aba9-a0685bbb57e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:13.907369 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437e5882-6db6-4c30-aba9-a0685bbb57e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:13.908873 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437e5882-6db6-4c30-aba9-a0685bbb57e7 service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:13.910948 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1310/2626] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:28:13 2022] POST /identity/v3/users => generated 299 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:28:13.915048 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e9356c-725a-4d7a-91c1-659e44522e16 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:13.921336 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1317/2627] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:28:13.926097 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e9356c-725a-4d7a-91c1-659e44522e16 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:13.930950 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dee242-6bac-4f33-a47e-d391f54b9fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:13.931638 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dee242-6bac-4f33-a47e-d391f54b9fee admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:13.931928 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dee242-6bac-4f33-a47e-d391f54b9fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:13.932211 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dee242-6bac-4f33-a47e-d391f54b9fee admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:13.932941 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dee242-6bac-4f33-a47e-d391f54b9fee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:13.934592 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dee242-6bac-4f33-a47e-d391f54b9fee admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:13.948587 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1311/2628] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:28:13.950559 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e5882-6db6-4c30-aba9-a0685bbb57e7 service neutron] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:13.960639 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e5882-6db6-4c30-aba9-a0685bbb57e7 service neutron] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:13.966031 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64def074-acc4-4ff0-9f7f-e8e8079d8016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:13.966693 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64def074-acc4-4ff0-9f7f-e8e8079d8016 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:13.966992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64def074-acc4-4ff0-9f7f-e8e8079d8016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:13.967267 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64def074-acc4-4ff0-9f7f-e8e8079d8016 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:13.968041 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64def074-acc4-4ff0-9f7f-e8e8079d8016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:13.969714 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64def074-acc4-4ff0-9f7f-e8e8079d8016 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:13.980371 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1318/2629] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:28:13.991451 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dee242-6bac-4f33-a47e-d391f54b9fee admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:14.004091 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dee242-6bac-4f33-a47e-d391f54b9fee admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:14.009657 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45857cd-8472-42c7-81da-9f4c32854c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:14.010553 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45857cd-8472-42c7-81da-9f4c32854c43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:14.010836 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45857cd-8472-42c7-81da-9f4c32854c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:14.011110 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45857cd-8472-42c7-81da-9f4c32854c43 admin admin] PATH_INFO: `/v3/projects/fec7088104454472a44015a743c0c284/users/91dd46d2918645fe9a425d0f2a10d7d0/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:14.030522 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45857cd-8472-42c7-81da-9f4c32854c43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fec7088104454472a44015a743c0c284, user_id=91dd46d2918645fe9a425d0f2a10d7d0, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:14.033592 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45857cd-8472-42c7-81da-9f4c32854c43 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:14.050952 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1312/2630] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:28:13 2022] PUT /identity/v3/projects/fec7088104454472a44015a743c0c284/users/91dd46d2918645fe9a425d0f2a10d7d0/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:14.118402 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2979353-e222-45e7-925c-fb98e2aad2ca None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:14.118842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2979353-e222-45e7-925c-fb98e2aad2ca None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:14.119223 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2979353-e222-45e7-925c-fb98e2aad2ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:14.149051 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-a2979353-e222-45e7-925c-fb98e2aad2ca None None] MFA Rules not processed for user `91dd46d2918645fe9a425d0f2a10d7d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:14.150540 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-a2979353-e222-45e7-925c-fb98e2aad2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:14.242692 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1319/2631] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:14 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:14.255624 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45857cd-8472-42c7-81da-9f4c32854c43 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:14.260589 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a45857cd-8472-42c7-81da-9f4c32854c43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:14.364117 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45857cd-8472-42c7-81da-9f4c32854c43 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:14.374581 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ce3787-6f0a-4809-ba3c-643d1cb43821 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:14.375845 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce3787-6f0a-4809-ba3c-643d1cb43821 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:14.376559 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce3787-6f0a-4809-ba3c-643d1cb43821 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:14.377009 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce3787-6f0a-4809-ba3c-643d1cb43821 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:14.399941 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ce3787-6f0a-4809-ba3c-643d1cb43821 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:14.401505 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ce3787-6f0a-4809-ba3c-643d1cb43821 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:14.413726 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1313/2632] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:14 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:28:15.948521 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2979353-e222-45e7-925c-fb98e2aad2ca None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:15.950483 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-a2979353-e222-45e7-925c-fb98e2aad2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:15.956044 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:16.037982 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2979353-e222-45e7-925c-fb98e2aad2ca None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:16.041724 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b01a59-7bbb-46ea-8a0f-cf15e3ee4675 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:16.043110 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b01a59-7bbb-46ea-8a0f-cf15e3ee4675 service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:16.043110 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b01a59-7bbb-46ea-8a0f-cf15e3ee4675 service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:16.043715 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b01a59-7bbb-46ea-8a0f-cf15e3ee4675 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:16.046184 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-29b01a59-7bbb-46ea-8a0f-cf15e3ee4675 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:16.134010 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b01a59-7bbb-46ea-8a0f-cf15e3ee4675 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:16.135970 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b01a59-7bbb-46ea-8a0f-cf15e3ee4675 service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:16.150325 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1320/2633] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:15 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:28:20.496276 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ce3787-6f0a-4809-ba3c-643d1cb43821 service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:20.497925 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-23ce3787-6f0a-4809-ba3c-643d1cb43821 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:20.577806 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ce3787-6f0a-4809-ba3c-643d1cb43821 service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:20.586112 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7087c5-6ca0-4545-94b3-9be3f3592bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:20.589088 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7087c5-6ca0-4545-94b3-9be3f3592bdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:20.589573 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7087c5-6ca0-4545-94b3-9be3f3592bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:20.589877 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7087c5-6ca0-4545-94b3-9be3f3592bdf admin admin] PATH_INFO: `/v3/users/5e052654361f4580ab36a347610f63c4` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:20.591396 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7087c5-6ca0-4545-94b3-9be3f3592bdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e052654361f4580ab36a347610f63c4)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:20.596718 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7087c5-6ca0-4545-94b3-9be3f3592bdf admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:20.601392 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:20.669001 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1d7087c5-6ca0-4545-94b3-9be3f3592bdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e052654361f4580ab36a347610f63c4'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:20.671067 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1d7087c5-6ca0-4545-94b3-9be3f3592bdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e052654361f4580ab36a347610f63c4'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:20.685999 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1d7087c5-6ca0-4545-94b3-9be3f3592bdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e052654361f4580ab36a347610f63c4'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:20.695628 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1d7087c5-6ca0-4545-94b3-9be3f3592bdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e052654361f4580ab36a347610f63c4'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:20.710026 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1d7087c5-6ca0-4545-94b3-9be3f3592bdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e052654361f4580ab36a347610f63c4'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:20.718271 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1314/2634] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:28:20 2022] DELETE /identity/v3/users/5e052654361f4580ab36a347610f63c4 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:21.130571 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b01a59-7bbb-46ea-8a0f-cf15e3ee4675 service neutron] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:21.142753 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b01a59-7bbb-46ea-8a0f-cf15e3ee4675 service neutron] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:21.208320 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c525ce47-d955-4c15-a084-97caa4f75276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:21.209207 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525ce47-d955-4c15-a084-97caa4f75276 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:21.209493 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525ce47-d955-4c15-a084-97caa4f75276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:21.209779 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c525ce47-d955-4c15-a084-97caa4f75276 admin admin] PATH_INFO: `/v3/projects/29d1268531c344eaaf28dc6c1498a467` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:21.211030 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c525ce47-d955-4c15-a084-97caa4f75276 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29d1268531c344eaaf28dc6c1498a467)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:21.213087 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c525ce47-d955-4c15-a084-97caa4f75276 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:21.266416 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-c525ce47-d955-4c15-a084-97caa4f75276 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29d1268531c344eaaf28dc6c1498a467', 'request_id': 'req-c525ce47-d955-4c15-a084-97caa4f75276'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:21.273127 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-c525ce47-d955-4c15-a084-97caa4f75276 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29d1268531c344eaaf28dc6c1498a467', 'request_id': 'req-c525ce47-d955-4c15-a084-97caa4f75276'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:21.275074 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:21.283977 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-c525ce47-d955-4c15-a084-97caa4f75276 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29d1268531c344eaaf28dc6c1498a467', 'request_id': 'req-c525ce47-d955-4c15-a084-97caa4f75276'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:21.296563 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-c525ce47-d955-4c15-a084-97caa4f75276 admin admin] The token cache is being invalidate because project 29d1268531c344eaaf28dc6c1498a467 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:28:21.296947 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-c525ce47-d955-4c15-a084-97caa4f75276 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:21.297656 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-c525ce47-d955-4c15-a084-97caa4f75276 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:21.299511 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1321/2635] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:28:21 2022] DELETE /identity/v3/projects/29d1268531c344eaaf28dc6c1498a467 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:21.304416 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440990b0-e1f0-4c13-b374-f2b414a13a57 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:21.304716 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440990b0-e1f0-4c13-b374-f2b414a13a57 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:21.304998 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440990b0-e1f0-4c13-b374-f2b414a13a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:21.322884 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-440990b0-e1f0-4c13-b374-f2b414a13a57 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:21.324176 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-440990b0-e1f0-4c13-b374-f2b414a13a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:21.390989 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1315/2636] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:21.399496 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb6de3e-f558-4869-888c-dea7c0672845 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:21.400182 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb6de3e-f558-4869-888c-dea7c0672845 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:21.400734 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb6de3e-f558-4869-888c-dea7c0672845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:21.427782 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-7eb6de3e-f558-4869-888c-dea7c0672845 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:21.429139 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-7eb6de3e-f558-4869-888c-dea7c0672845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:21.438572 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1322/2637] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:21.442188 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440990b0-e1f0-4c13-b374-f2b414a13a57 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:21.453296 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440990b0-e1f0-4c13-b374-f2b414a13a57 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:21.460082 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f7b40-c201-4550-9259-5da895c8c4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:21.460748 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f7b40-c201-4550-9259-5da895c8c4fe admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:21.461034 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f7b40-c201-4550-9259-5da895c8c4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:21.461314 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f7b40-c201-4550-9259-5da895c8c4fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:21.462108 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f7b40-c201-4550-9259-5da895c8c4fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:28:21.462365 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f7b40-c201-4550-9259-5da895c8c4fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:21.464012 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f7b40-c201-4550-9259-5da895c8c4fe admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:21.481556 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1316/2638] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:28:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:28:21.482524 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb6de3e-f558-4869-888c-dea7c0672845 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:21.492513 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb6de3e-f558-4869-888c-dea7c0672845 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:21.501765 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b2f53-492b-4329-9016-ab6a9134634b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:21.502451 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b2f53-492b-4329-9016-ab6a9134634b admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:21.502734 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b2f53-492b-4329-9016-ab6a9134634b admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:21.503011 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b2f53-492b-4329-9016-ab6a9134634b admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:21.503774 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16b2f53-492b-4329-9016-ab6a9134634b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:21.506056 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16b2f53-492b-4329-9016-ab6a9134634b admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:21.563115 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f7b40-c201-4550-9259-5da895c8c4fe admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:21.564232 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1323/2639] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:28:21 2022] POST /identity/v3/projects => generated 365 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:28:21.580807 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f7b40-c201-4550-9259-5da895c8c4fe admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:21.738584 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ff365-1244-4910-b93a-838406f76f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:21.739312 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ff365-1244-4910-b93a-838406f76f33 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:21.739639 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ff365-1244-4910-b93a-838406f76f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:21.739928 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ff365-1244-4910-b93a-838406f76f33 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:21.747893 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4ff365-1244-4910-b93a-838406f76f33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:21.751366 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4ff365-1244-4910-b93a-838406f76f33 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:21.778178 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:21.803104 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1317/2640] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:28:21 2022] POST /identity/v3/users => generated 283 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:28:21.807670 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b2f53-492b-4329-9016-ab6a9134634b admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:21.827580 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16b2f53-492b-4329-9016-ab6a9134634b admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:21.853417 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c21830-4dba-49ae-92dc-95dd43556a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:21.854210 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c21830-4dba-49ae-92dc-95dd43556a1c admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:21.854708 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c21830-4dba-49ae-92dc-95dd43556a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:21.855039 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c21830-4dba-49ae-92dc-95dd43556a1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:21.856724 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c21830-4dba-49ae-92dc-95dd43556a1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:21.858723 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c21830-4dba-49ae-92dc-95dd43556a1c admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:21.888720 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1324/2641] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:21 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:28:21.893072 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ff365-1244-4910-b93a-838406f76f33 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:21.909301 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ff365-1244-4910-b93a-838406f76f33 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:21.915698 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd2922-0b2d-45c3-a54f-9d0703a9c22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:21.916605 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd2922-0b2d-45c3-a54f-9d0703a9c22a admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:21.917074 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd2922-0b2d-45c3-a54f-9d0703a9c22a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:21.917511 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd2922-0b2d-45c3-a54f-9d0703a9c22a admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:21.918961 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd2922-0b2d-45c3-a54f-9d0703a9c22a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:21.921358 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd2922-0b2d-45c3-a54f-9d0703a9c22a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:21.949018 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1318/2642] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:21 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:28:21.952964 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c21830-4dba-49ae-92dc-95dd43556a1c admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:21.964393 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c21830-4dba-49ae-92dc-95dd43556a1c admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:21.970019 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df435ebb-a2b4-45e2-aad5-bd1c97a9b6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:21.971006 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df435ebb-a2b4-45e2-aad5-bd1c97a9b6ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:21.971354 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df435ebb-a2b4-45e2-aad5-bd1c97a9b6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:21.971719 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df435ebb-a2b4-45e2-aad5-bd1c97a9b6ae admin admin] PATH_INFO: `/v3/projects/e900e3c84f394cdd9653999f94084fda/users/350a76fe0de54add881ff71750fcbaf8/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:22.004946 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df435ebb-a2b4-45e2-aad5-bd1c97a9b6ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=e900e3c84f394cdd9653999f94084fda, user_id=350a76fe0de54add881ff71750fcbaf8, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:22.008315 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df435ebb-a2b4-45e2-aad5-bd1c97a9b6ae admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:22.041145 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1325/2643] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:28:21 2022] PUT /identity/v3/projects/e900e3c84f394cdd9653999f94084fda/users/350a76fe0de54add881ff71750fcbaf8/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:22.054072 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84cd2e-3dca-4526-a889-1d4006369388 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:22.057195 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84cd2e-3dca-4526-a889-1d4006369388 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:22.058416 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d84cd2e-3dca-4526-a889-1d4006369388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:22.100401 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-3d84cd2e-3dca-4526-a889-1d4006369388 None None] MFA Rules not processed for user `350a76fe0de54add881ff71750fcbaf8`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:22.102510 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3d84cd2e-3dca-4526-a889-1d4006369388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:22.240599 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1319/2644] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:22 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:22.245429 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df435ebb-a2b4-45e2-aad5-bd1c97a9b6ae admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:22.390940 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df435ebb-a2b4-45e2-aad5-bd1c97a9b6ae admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:22.406232 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8eed78-4c5a-4b6a-8938-35700bf753d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:22.406929 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8eed78-4c5a-4b6a-8938-35700bf753d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:22.407214 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8eed78-4c5a-4b6a-8938-35700bf753d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:22.407490 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8eed78-4c5a-4b6a-8938-35700bf753d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:22.411761 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8eed78-4c5a-4b6a-8938-35700bf753d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:22.413784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8eed78-4c5a-4b6a-8938-35700bf753d3 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:22.450714 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1326/2645] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:28:22 2022] POST /identity/v3/projects => generated 363 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:28:22.455875 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84cd2e-3dca-4526-a889-1d4006369388 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:22.471522 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d84cd2e-3dca-4526-a889-1d4006369388 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:22.551988 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd9ddf-ac05-46cf-b8e6-b49392fe8ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:22.552778 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd9ddf-ac05-46cf-b8e6-b49392fe8ec3 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:22.553163 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd9ddf-ac05-46cf-b8e6-b49392fe8ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:22.553719 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd9ddf-ac05-46cf-b8e6-b49392fe8ec3 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:22.554525 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd9ddf-ac05-46cf-b8e6-b49392fe8ec3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:22.556610 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd9ddf-ac05-46cf-b8e6-b49392fe8ec3 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:22.588892 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1320/2646] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:28:22 2022] POST /identity/v3/users => generated 288 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:28:22.591209 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8eed78-4c5a-4b6a-8938-35700bf753d3 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:22.602675 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8eed78-4c5a-4b6a-8938-35700bf753d3 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:22.609417 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a331f4-4c39-4b2c-abef-26403f66991f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:22.610106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a331f4-4c39-4b2c-abef-26403f66991f admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:22.610387 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a331f4-4c39-4b2c-abef-26403f66991f admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:22.610661 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a331f4-4c39-4b2c-abef-26403f66991f admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:22.611403 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a331f4-4c39-4b2c-abef-26403f66991f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:22.619682 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a331f4-4c39-4b2c-abef-26403f66991f admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:22.630694 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1327/2647] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:28:22.631823 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd9ddf-ac05-46cf-b8e6-b49392fe8ec3 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:22.645527 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd9ddf-ac05-46cf-b8e6-b49392fe8ec3 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:22.659811 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a10993-4d06-4b66-a8e3-77ec79209c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:22.661226 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a10993-4d06-4b66-a8e3-77ec79209c7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:22.661576 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a10993-4d06-4b66-a8e3-77ec79209c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:22.661961 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a10993-4d06-4b66-a8e3-77ec79209c7d admin admin] PATH_INFO: `/v3/projects/83931a19de3c4a6ab3af4824a800b85b/users/b8f466a8fc23477ea4c98a09631c71e7/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:22.709963 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a10993-4d06-4b66-a8e3-77ec79209c7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=83931a19de3c4a6ab3af4824a800b85b, user_id=b8f466a8fc23477ea4c98a09631c71e7, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:22.713049 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a10993-4d06-4b66-a8e3-77ec79209c7d admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:22.739063 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1321/2648] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:28:22 2022] PUT /identity/v3/projects/83931a19de3c4a6ab3af4824a800b85b/users/b8f466a8fc23477ea4c98a09631c71e7/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:28:22.742952 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a331f4-4c39-4b2c-abef-26403f66991f admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:22.760198 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a331f4-4c39-4b2c-abef-26403f66991f admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:22.841802 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:22.862486 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc24bac-264b-44f5-b24a-d01c485b9d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:22.863698 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc24bac-264b-44f5-b24a-d01c485b9d50 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:22.864363 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc24bac-264b-44f5-b24a-d01c485b9d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:22.864754 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc24bac-264b-44f5-b24a-d01c485b9d50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:22.866166 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc24bac-264b-44f5-b24a-d01c485b9d50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:22.869255 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc24bac-264b-44f5-b24a-d01c485b9d50 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:22.887652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1328/2649] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:28:22.890406 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a10993-4d06-4b66-a8e3-77ec79209c7d admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:22.900952 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a10993-4d06-4b66-a8e3-77ec79209c7d admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:22.906971 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f3890e-ba57-4a02-b760-b9bc712f7e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:22.908856 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f3890e-ba57-4a02-b760-b9bc712f7e5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:22.908856 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f3890e-ba57-4a02-b760-b9bc712f7e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:22.908856 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f3890e-ba57-4a02-b760-b9bc712f7e5b admin admin] PATH_INFO: `/v3/projects/83931a19de3c4a6ab3af4824a800b85b/users/b8f466a8fc23477ea4c98a09631c71e7/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:22.911192 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f3890e-ba57-4a02-b760-b9bc712f7e5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=83931a19de3c4a6ab3af4824a800b85b, user_id=b8f466a8fc23477ea4c98a09631c71e7, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:22.914593 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f3890e-ba57-4a02-b760-b9bc712f7e5b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:22.929958 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1322/2650] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:28:22 2022] PUT /identity/v3/projects/83931a19de3c4a6ab3af4824a800b85b/users/b8f466a8fc23477ea4c98a09631c71e7/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:22.941252 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8318a16-25f9-4615-8f25-0f5f9283dcab None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:22.941677 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8318a16-25f9-4615-8f25-0f5f9283dcab None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:22.942109 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8318a16-25f9-4615-8f25-0f5f9283dcab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:22.974999 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-f8318a16-25f9-4615-8f25-0f5f9283dcab None None] MFA Rules not processed for user `b8f466a8fc23477ea4c98a09631c71e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:22.976331 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-f8318a16-25f9-4615-8f25-0f5f9283dcab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:23.068182 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1329/2651] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:22 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:23.082248 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f3890e-ba57-4a02-b760-b9bc712f7e5b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:23.084996 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-62f3890e-ba57-4a02-b760-b9bc712f7e5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:23.129443 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:23.165254 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f3890e-ba57-4a02-b760-b9bc712f7e5b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:23.169493 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a0170c-461d-4c50-9c2f-d5726bfc148e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:23.170514 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0170c-461d-4c50-9c2f-d5726bfc148e service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:23.170514 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0170c-461d-4c50-9c2f-d5726bfc148e service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:23.170720 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a0170c-461d-4c50-9c2f-d5726bfc148e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:23.182155 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a0170c-461d-4c50-9c2f-d5726bfc148e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:23.184453 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a0170c-461d-4c50-9c2f-d5726bfc148e service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:23.195735 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1323/2652] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:23 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:28:23.273688 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8318a16-25f9-4615-8f25-0f5f9283dcab None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:23.284214 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8318a16-25f9-4615-8f25-0f5f9283dcab None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:23.288075 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf493e-1835-4455-a1f2-39d59858c08c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:23.288738 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf493e-1835-4455-a1f2-39d59858c08c service cinder] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:23.289018 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf493e-1835-4455-a1f2-39d59858c08c service cinder] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:23.289289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf493e-1835-4455-a1f2-39d59858c08c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:23.362447 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bf493e-1835-4455-a1f2-39d59858c08c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:23.364495 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bf493e-1835-4455-a1f2-39d59858c08c service cinder] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:23.377858 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1330/2653] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:23 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:28:23.857032 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a0170c-461d-4c50-9c2f-d5726bfc148e service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:23.858506 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a3a0170c-461d-4c50-9c2f-d5726bfc148e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:23.919433 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf493e-1835-4455-a1f2-39d59858c08c service cinder] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:23.925521 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-f4bf493e-1835-4455-a1f2-39d59858c08c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:23.949032 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a0170c-461d-4c50-9c2f-d5726bfc148e service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:23.954571 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e78e5b-42da-4478-8773-c683d4684ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:23.955678 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e78e5b-42da-4478-8773-c683d4684ca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:23.955997 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e78e5b-42da-4478-8773-c683d4684ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:23.956431 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e78e5b-42da-4478-8773-c683d4684ca9 admin admin] PATH_INFO: `/v3/users/91dd46d2918645fe9a425d0f2a10d7d0` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:23.958093 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e78e5b-42da-4478-8773-c683d4684ca9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91dd46d2918645fe9a425d0f2a10d7d0)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:23.960520 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e78e5b-42da-4478-8773-c683d4684ca9 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:24.039134 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-10e78e5b-42da-4478-8773-c683d4684ca9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91dd46d2918645fe9a425d0f2a10d7d0'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:24.040722 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-10e78e5b-42da-4478-8773-c683d4684ca9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91dd46d2918645fe9a425d0f2a10d7d0'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:24.054595 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf493e-1835-4455-a1f2-39d59858c08c service cinder] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:24.060756 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cee83b-f502-4012-a694-9c9d61b7c8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:24.062837 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cee83b-f502-4012-a694-9c9d61b7c8f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:24.062837 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cee83b-f502-4012-a694-9c9d61b7c8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:24.062837 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cee83b-f502-4012-a694-9c9d61b7c8f2 admin admin] PATH_INFO: `/v3/users/e708995dd77c4e5cb9430286bf5991a9` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:24.063652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cee83b-f502-4012-a694-9c9d61b7c8f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e708995dd77c4e5cb9430286bf5991a9)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:24.066303 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cee83b-f502-4012-a694-9c9d61b7c8f2 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:24.070094 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-10e78e5b-42da-4478-8773-c683d4684ca9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91dd46d2918645fe9a425d0f2a10d7d0'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:24.085108 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:24.094504 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-10e78e5b-42da-4478-8773-c683d4684ca9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91dd46d2918645fe9a425d0f2a10d7d0'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:24.121120 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-10e78e5b-42da-4478-8773-c683d4684ca9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91dd46d2918645fe9a425d0f2a10d7d0'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:24.136008 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1324/2654] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:28:23 2022] DELETE /identity/v3/users/91dd46d2918645fe9a425d0f2a10d7d0 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:28:24.141228 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a925dab-e1ac-452b-a59e-9c3e47648ca6 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:24.141795 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a925dab-e1ac-452b-a59e-9c3e47648ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:24.142233 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a925dab-e1ac-452b-a59e-9c3e47648ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:24.149823 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-f5cee83b-f502-4012-a694-9c9d61b7c8f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e708995dd77c4e5cb9430286bf5991a9'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:24.150717 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-f5cee83b-f502-4012-a694-9c9d61b7c8f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e708995dd77c4e5cb9430286bf5991a9'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:24.167883 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-f5cee83b-f502-4012-a694-9c9d61b7c8f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e708995dd77c4e5cb9430286bf5991a9'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:24.179655 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-0a925dab-e1ac-452b-a59e-9c3e47648ca6 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:24.181039 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-f5cee83b-f502-4012-a694-9c9d61b7c8f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e708995dd77c4e5cb9430286bf5991a9'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:24.181369 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0a925dab-e1ac-452b-a59e-9c3e47648ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:24.181774 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-f5cee83b-f502-4012-a694-9c9d61b7c8f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e708995dd77c4e5cb9430286bf5991a9'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:24.208011 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1331/2655] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:28:23 2022] DELETE /identity/v3/users/e708995dd77c4e5cb9430286bf5991a9 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:24.212445 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa223d40-4bc2-4363-a2ac-1cd92d6c7222 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:24.212845 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa223d40-4bc2-4363-a2ac-1cd92d6c7222 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:24.213340 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa223d40-4bc2-4363-a2ac-1cd92d6c7222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:24.241827 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-fa223d40-4bc2-4363-a2ac-1cd92d6c7222 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:24.243426 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-fa223d40-4bc2-4363-a2ac-1cd92d6c7222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:24.245970 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:24.270410 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1325/2656] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:24.283497 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a925dab-e1ac-452b-a59e-9c3e47648ca6 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:24.285533 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0a925dab-e1ac-452b-a59e-9c3e47648ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:24.345981 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1332/2657] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:24.358582 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a925dab-e1ac-452b-a59e-9c3e47648ca6 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:24.364766 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa223d40-4bc2-4363-a2ac-1cd92d6c7222 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:24.366471 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-fa223d40-4bc2-4363-a2ac-1cd92d6c7222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:24.448574 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c52410-a74c-4a8e-949b-b423458c97d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:24.449095 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c52410-a74c-4a8e-949b-b423458c97d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:24.449386 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c52410-a74c-4a8e-949b-b423458c97d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:24.450215 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c52410-a74c-4a8e-949b-b423458c97d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:24.472592 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa223d40-4bc2-4363-a2ac-1cd92d6c7222 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:24.478490 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd23d9-d05a-4d3e-b3f0-cc34a4c7bf3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:24.479479 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd23d9-d05a-4d3e-b3f0-cc34a4c7bf3b service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:24.479971 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd23d9-d05a-4d3e-b3f0-cc34a4c7bf3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:24.480449 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd23d9-d05a-4d3e-b3f0-cc34a4c7bf3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:24.485270 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c52410-a74c-4a8e-949b-b423458c97d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:24.486776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c52410-a74c-4a8e-949b-b423458c97d8 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:24.499402 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cd23d9-d05a-4d3e-b3f0-cc34a4c7bf3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:24.500203 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1326/2658] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:28:24.501872 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cd23d9-d05a-4d3e-b3f0-cc34a4c7bf3b service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:24.546090 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1333/2659] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:28:24.898423 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c52410-a74c-4a8e-949b-b423458c97d8 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:24.900628 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-23c52410-a74c-4a8e-949b-b423458c97d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:24.921953 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c52410-a74c-4a8e-949b-b423458c97d8 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:24.930393 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcae0b1-b650-4d41-ac47-a31de65d08d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:24.931785 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcae0b1-b650-4d41-ac47-a31de65d08d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:24.932329 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcae0b1-b650-4d41-ac47-a31de65d08d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:24.932901 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcae0b1-b650-4d41-ac47-a31de65d08d7 admin admin] PATH_INFO: `/v3/users/c3871ceed47e4fcea07c5f879c8462cb` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:24.935270 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcae0b1-b650-4d41-ac47-a31de65d08d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3871ceed47e4fcea07c5f879c8462cb)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:24.938194 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcae0b1-b650-4d41-ac47-a31de65d08d7 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:24.981909 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd23d9-d05a-4d3e-b3f0-cc34a4c7bf3b service neutron] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:24.984306 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-57cd23d9-d05a-4d3e-b3f0-cc34a4c7bf3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:25.004107 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd23d9-d05a-4d3e-b3f0-cc34a4c7bf3b service neutron] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:25.015005 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7bcae0b1-b650-4d41-ac47-a31de65d08d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3871ceed47e4fcea07c5f879c8462cb'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:25.015793 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7bcae0b1-b650-4d41-ac47-a31de65d08d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3871ceed47e4fcea07c5f879c8462cb'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:25.023232 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1071b729-12c9-4bca-a029-07a80f658529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:25.024187 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1071b729-12c9-4bca-a029-07a80f658529 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:25.024465 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1071b729-12c9-4bca-a029-07a80f658529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:25.024877 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1071b729-12c9-4bca-a029-07a80f658529 admin admin] PATH_INFO: `/v3/projects/fec7088104454472a44015a743c0c284` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:25.026091 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1071b729-12c9-4bca-a029-07a80f658529 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fec7088104454472a44015a743c0c284)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:25.028719 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1071b729-12c9-4bca-a029-07a80f658529 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:25.048558 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7bcae0b1-b650-4d41-ac47-a31de65d08d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3871ceed47e4fcea07c5f879c8462cb'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:25.059697 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7bcae0b1-b650-4d41-ac47-a31de65d08d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3871ceed47e4fcea07c5f879c8462cb'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:25.077728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7bcae0b1-b650-4d41-ac47-a31de65d08d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3871ceed47e4fcea07c5f879c8462cb'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:25.088349 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1327/2660] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:28:24 2022] DELETE /identity/v3/users/c3871ceed47e4fcea07c5f879c8462cb => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:25.097464 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:25.117871 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-1071b729-12c9-4bca-a029-07a80f658529 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fec7088104454472a44015a743c0c284', 'request_id': 'req-1071b729-12c9-4bca-a029-07a80f658529'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:25.125687 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-1071b729-12c9-4bca-a029-07a80f658529 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fec7088104454472a44015a743c0c284', 'request_id': 'req-1071b729-12c9-4bca-a029-07a80f658529'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:25.136575 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-1071b729-12c9-4bca-a029-07a80f658529 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fec7088104454472a44015a743c0c284', 'request_id': 'req-1071b729-12c9-4bca-a029-07a80f658529'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:25.164602 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-1071b729-12c9-4bca-a029-07a80f658529 admin admin] The token cache is being invalidate because project fec7088104454472a44015a743c0c284 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:28:25.165161 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-1071b729-12c9-4bca-a029-07a80f658529 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:25.167417 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-1071b729-12c9-4bca-a029-07a80f658529 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:25.180961 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e86b7c-7a91-4660-beee-a3fdf0ca45f8 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:25.181259 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e86b7c-7a91-4660-beee-a3fdf0ca45f8 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:25.181579 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e86b7c-7a91-4660-beee-a3fdf0ca45f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:25.182981 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1334/2661] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:28:24 2022] DELETE /identity/v3/projects/fec7088104454472a44015a743c0c284 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:25.199987 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1071b729-12c9-4bca-a029-07a80f658529 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:25.201759 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-1071b729-12c9-4bca-a029-07a80f658529 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:25.208446 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-78e86b7c-7a91-4660-beee-a3fdf0ca45f8 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:25.209679 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-78e86b7c-7a91-4660-beee-a3fdf0ca45f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:25.234875 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:25.289702 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1328/2662] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:25.294610 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1071b729-12c9-4bca-a029-07a80f658529 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:25.298063 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34b71b-6f14-4001-81ce-2a60fe169262 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:25.298491 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34b71b-6f14-4001-81ce-2a60fe169262 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:25.298799 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34b71b-6f14-4001-81ce-2a60fe169262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:25.302852 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fba7f9-a87d-4f0d-9e33-92d46a6321e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:25.303858 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fba7f9-a87d-4f0d-9e33-92d46a6321e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:25.304219 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fba7f9-a87d-4f0d-9e33-92d46a6321e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:25.304616 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fba7f9-a87d-4f0d-9e33-92d46a6321e6 admin admin] PATH_INFO: `/v3/projects/705fba370d7a42b18e24d3772bcccc0e` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:25.305853 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fba7f9-a87d-4f0d-9e33-92d46a6321e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=705fba370d7a42b18e24d3772bcccc0e)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:25.308100 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fba7f9-a87d-4f0d-9e33-92d46a6321e6 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:25.323395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-6b34b71b-6f14-4001-81ce-2a60fe169262 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:25.324635 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6b34b71b-6f14-4001-81ce-2a60fe169262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:25.333075 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1329/2663] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:25.337142 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34b71b-6f14-4001-81ce-2a60fe169262 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:25.347320 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34b71b-6f14-4001-81ce-2a60fe169262 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:25.352418 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6b37f-8ad7-4da3-9a11-8e00e9a2f4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:25.353107 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6b37f-8ad7-4da3-9a11-8e00e9a2f4e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:25.353379 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6b37f-8ad7-4da3-9a11-8e00e9a2f4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:25.353658 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa6b37f-8ad7-4da3-9a11-8e00e9a2f4e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:25.354684 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa6b37f-8ad7-4da3-9a11-8e00e9a2f4e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:28:25.359985 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa6b37f-8ad7-4da3-9a11-8e00e9a2f4e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:25.362285 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa6b37f-8ad7-4da3-9a11-8e00e9a2f4e2 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:25.372034 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-f2fba7f9-a87d-4f0d-9e33-92d46a6321e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '705fba370d7a42b18e24d3772bcccc0e', 'request_id': 'req-f2fba7f9-a87d-4f0d-9e33-92d46a6321e6'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:25.379817 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-f2fba7f9-a87d-4f0d-9e33-92d46a6321e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '705fba370d7a42b18e24d3772bcccc0e', 'request_id': 'req-f2fba7f9-a87d-4f0d-9e33-92d46a6321e6'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:25.389506 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1330/2664] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:28:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:28:25.389506 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6b37f-8ad7-4da3-9a11-8e00e9a2f4e2 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:25.389838 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-f2fba7f9-a87d-4f0d-9e33-92d46a6321e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '705fba370d7a42b18e24d3772bcccc0e', 'request_id': 'req-f2fba7f9-a87d-4f0d-9e33-92d46a6321e6'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:25.392336 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-faa6b37f-8ad7-4da3-9a11-8e00e9a2f4e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:25.400801 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-f2fba7f9-a87d-4f0d-9e33-92d46a6321e6 admin admin] The token cache is being invalidate because project 705fba370d7a42b18e24d3772bcccc0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:28:25.401092 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-f2fba7f9-a87d-4f0d-9e33-92d46a6321e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:25.401748 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-f2fba7f9-a87d-4f0d-9e33-92d46a6321e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:25.404267 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1335/2665] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:28:25 2022] DELETE /identity/v3/projects/705fba370d7a42b18e24d3772bcccc0e => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:28:25.405001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fba7f9-a87d-4f0d-9e33-92d46a6321e6 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:25.406071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa6b37f-8ad7-4da3-9a11-8e00e9a2f4e2 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:25.406292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-f2fba7f9-a87d-4f0d-9e33-92d46a6321e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:25.456484 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:25.463840 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ff3b8-a38f-45dc-959f-66ad173bde2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:25.464723 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ff3b8-a38f-45dc-959f-66ad173bde2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:25.465008 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ff3b8-a38f-45dc-959f-66ad173bde2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:25.465850 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ff3b8-a38f-45dc-959f-66ad173bde2c admin admin] PATH_INFO: `/v3/projects/d631f17810f94dcc91dfcb28a4b17c9a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:25.467063 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ff3b8-a38f-45dc-959f-66ad173bde2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d631f17810f94dcc91dfcb28a4b17c9a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:25.469011 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ff3b8-a38f-45dc-959f-66ad173bde2c admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:25.474812 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fba7f9-a87d-4f0d-9e33-92d46a6321e6 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:25.479926 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d77568-ea3b-47a4-b1e2-299620d68313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:25.480588 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d77568-ea3b-47a4-b1e2-299620d68313 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:25.480956 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d77568-ea3b-47a4-b1e2-299620d68313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:25.481472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d77568-ea3b-47a4-b1e2-299620d68313 admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:25.481915 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d77568-ea3b-47a4-b1e2-299620d68313 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:25.483913 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d77568-ea3b-47a4-b1e2-299620d68313 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:25.511157 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1336/2666] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:28:25 2022] POST /identity/v3/projects => generated 369 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:28:25.513484 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61be9c00-f9b9-425b-8674-ace5b3fce292 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:25.513827 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61be9c00-f9b9-425b-8674-ace5b3fce292 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:25.514108 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61be9c00-f9b9-425b-8674-ace5b3fce292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:25.525577 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-bb2ff3b8-a38f-45dc-959f-66ad173bde2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd631f17810f94dcc91dfcb28a4b17c9a', 'request_id': 'req-bb2ff3b8-a38f-45dc-959f-66ad173bde2c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:25.532724 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-bb2ff3b8-a38f-45dc-959f-66ad173bde2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd631f17810f94dcc91dfcb28a4b17c9a', 'request_id': 'req-bb2ff3b8-a38f-45dc-959f-66ad173bde2c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:25.533632 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-61be9c00-f9b9-425b-8674-ace5b3fce292 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:25.534856 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-61be9c00-f9b9-425b-8674-ace5b3fce292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:25.540610 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-bb2ff3b8-a38f-45dc-959f-66ad173bde2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd631f17810f94dcc91dfcb28a4b17c9a', 'request_id': 'req-bb2ff3b8-a38f-45dc-959f-66ad173bde2c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:25.551910 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-bb2ff3b8-a38f-45dc-959f-66ad173bde2c admin admin] The token cache is being invalidate because project d631f17810f94dcc91dfcb28a4b17c9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:28:25.552192 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-bb2ff3b8-a38f-45dc-959f-66ad173bde2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:25.553003 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-bb2ff3b8-a38f-45dc-959f-66ad173bde2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:25.554576 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1331/2667] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:28:25 2022] DELETE /identity/v3/projects/d631f17810f94dcc91dfcb28a4b17c9a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:25.555312 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ff3b8-a38f-45dc-959f-66ad173bde2c admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:25.556694 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-bb2ff3b8-a38f-45dc-959f-66ad173bde2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:25.591982 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:25.601224 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1337/2668] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:25.602072 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61be9c00-f9b9-425b-8674-ace5b3fce292 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:25.603439 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-61be9c00-f9b9-425b-8674-ace5b3fce292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:25.616524 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61be9c00-f9b9-425b-8674-ace5b3fce292 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:25.621191 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8bfec2-0175-481c-b27c-0502cf93be35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:25.622086 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8bfec2-0175-481c-b27c-0502cf93be35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:25.622373 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8bfec2-0175-481c-b27c-0502cf93be35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:25.622655 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8bfec2-0175-481c-b27c-0502cf93be35 admin admin] PATH_INFO: `/v3/users/2453702c7e11464d8aefea3614549ae9` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:25.630691 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ff3b8-a38f-45dc-959f-66ad173bde2c admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:25.636020 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e9d86-a0c7-46f3-aae6-2c17e87fb071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:25.636689 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e9d86-a0c7-46f3-aae6-2c17e87fb071 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:25.637492 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e9d86-a0c7-46f3-aae6-2c17e87fb071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:25.637610 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8bfec2-0175-481c-b27c-0502cf93be35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2453702c7e11464d8aefea3614549ae9)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:25.637769 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e9d86-a0c7-46f3-aae6-2c17e87fb071 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:25.638457 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e9d86-a0c7-46f3-aae6-2c17e87fb071 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:25.639484 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8bfec2-0175-481c-b27c-0502cf93be35 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:25.640419 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e9d86-a0c7-46f3-aae6-2c17e87fb071 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:25.661730 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1332/2669] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:28:25 2022] POST /identity/v3/users => generated 285 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:28:25.663004 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b36ae5-b39a-4417-84a0-1eedb2e72519 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:25.663291 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b36ae5-b39a-4417-84a0-1eedb2e72519 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:25.663586 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b36ae5-b39a-4417-84a0-1eedb2e72519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:25.683253 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c8b36ae5-b39a-4417-84a0-1eedb2e72519 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:25.684639 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c8b36ae5-b39a-4417-84a0-1eedb2e72519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:25.690137 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-dc8bfec2-0175-481c-b27c-0502cf93be35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2453702c7e11464d8aefea3614549ae9'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:25.690810 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-dc8bfec2-0175-481c-b27c-0502cf93be35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2453702c7e11464d8aefea3614549ae9'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:25.693026 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1333/2670] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:25.693799 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b36ae5-b39a-4417-84a0-1eedb2e72519 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:25.703923 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b36ae5-b39a-4417-84a0-1eedb2e72519 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:25.705017 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-dc8bfec2-0175-481c-b27c-0502cf93be35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2453702c7e11464d8aefea3614549ae9'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:25.708739 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1164a-1787-4224-b390-9023af30e3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:25.709320 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1164a-1787-4224-b390-9023af30e3b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:25.709605 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1164a-1787-4224-b390-9023af30e3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:25.709884 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1164a-1787-4224-b390-9023af30e3b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:25.710605 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb1164a-1787-4224-b390-9023af30e3b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:25.712143 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-dc8bfec2-0175-481c-b27c-0502cf93be35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2453702c7e11464d8aefea3614549ae9'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:25.712283 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb1164a-1787-4224-b390-9023af30e3b6 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:25.713141 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-dc8bfec2-0175-481c-b27c-0502cf93be35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2453702c7e11464d8aefea3614549ae9'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:25.723637 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1334/2671] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:28:25.724466 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1164a-1787-4224-b390-9023af30e3b6 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:25.726138 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ecb1164a-1787-4224-b390-9023af30e3b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:25.727820 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1338/2672] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:28:25 2022] DELETE /identity/v3/users/2453702c7e11464d8aefea3614549ae9 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:25.728856 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8bfec2-0175-481c-b27c-0502cf93be35 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:25.730177 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-dc8bfec2-0175-481c-b27c-0502cf93be35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:25.743199 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1164a-1787-4224-b390-9023af30e3b6 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:25.774062 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:25.810401 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a6520-2b19-4660-9ff9-b5f4c4957af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:25.810863 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8bfec2-0175-481c-b27c-0502cf93be35 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:25.811072 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a6520-2b19-4660-9ff9-b5f4c4957af1 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:25.811354 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a6520-2b19-4660-9ff9-b5f4c4957af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:25.811666 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a6520-2b19-4660-9ff9-b5f4c4957af1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:25.812396 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a6520-2b19-4660-9ff9-b5f4c4957af1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:28:25.812701 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a6520-2b19-4660-9ff9-b5f4c4957af1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:25.814521 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a6520-2b19-4660-9ff9-b5f4c4957af1 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:25.818571 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c500c-6f36-43fa-95f9-818fa494ed81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:25.819586 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c500c-6f36-43fa-95f9-818fa494ed81 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:25.820400 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c500c-6f36-43fa-95f9-818fa494ed81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:25.820876 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c500c-6f36-43fa-95f9-818fa494ed81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:25.822371 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212c500c-6f36-43fa-95f9-818fa494ed81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:25.825369 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212c500c-6f36-43fa-95f9-818fa494ed81 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:25.831081 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1335/2673] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:28:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:28:25.832141 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a6520-2b19-4660-9ff9-b5f4c4957af1 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:25.842342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a6520-2b19-4660-9ff9-b5f4c4957af1 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:25.842564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1339/2674] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:28:25.846929 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c500c-6f36-43fa-95f9-818fa494ed81 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:25.850326 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575e6ec-06df-4512-821d-557e6b7a82d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:25.851024 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575e6ec-06df-4512-821d-557e6b7a82d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:25.851308 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575e6ec-06df-4512-821d-557e6b7a82d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:25.851617 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575e6ec-06df-4512-821d-557e6b7a82d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:25.852310 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575e6ec-06df-4512-821d-557e6b7a82d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:25.854652 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575e6ec-06df-4512-821d-557e6b7a82d3 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:25.857295 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c500c-6f36-43fa-95f9-818fa494ed81 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:25.862817 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e293cc24-6607-4c4e-85e4-a32f49a35821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:25.863916 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293cc24-6607-4c4e-85e4-a32f49a35821 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:25.864200 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293cc24-6607-4c4e-85e4-a32f49a35821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:25.864472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293cc24-6607-4c4e-85e4-a32f49a35821 admin admin] PATH_INFO: `/v3/projects/d2f20deb401b4c348b0a18177ec0d2d7/users/e292de524b9c405dae3d520b60136155/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:25.884882 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e293cc24-6607-4c4e-85e4-a32f49a35821 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2f20deb401b4c348b0a18177ec0d2d7, user_id=e292de524b9c405dae3d520b60136155, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:25.888515 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e293cc24-6607-4c4e-85e4-a32f49a35821 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:25.889514 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1336/2675] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:28:25 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:28:25.893471 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575e6ec-06df-4512-821d-557e6b7a82d3 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:25.906004 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1340/2676] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:28:25 2022] PUT /identity/v3/projects/d2f20deb401b4c348b0a18177ec0d2d7/users/e292de524b9c405dae3d520b60136155/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:28:25.910914 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d889d-0b10-4288-a786-4e2ebaf4c86c None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:25.911237 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d889d-0b10-4288-a786-4e2ebaf4c86c None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:25.911518 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d889d-0b10-4288-a786-4e2ebaf4c86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:25.932185 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-d79d889d-0b10-4288-a786-4e2ebaf4c86c None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:25.933680 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-d79d889d-0b10-4288-a786-4e2ebaf4c86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:25.978905 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575e6ec-06df-4512-821d-557e6b7a82d3 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:25.998876 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:26.000919 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1341/2677] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:26.001699 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d889d-0b10-4288-a786-4e2ebaf4c86c None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:26.003012 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-d79d889d-0b10-4288-a786-4e2ebaf4c86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:26.016855 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d889d-0b10-4288-a786-4e2ebaf4c86c None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:26.022132 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d7330-2c47-4534-aca9-5861347597d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:26.023051 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d7330-2c47-4534-aca9-5861347597d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:26.023750 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d7330-2c47-4534-aca9-5861347597d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:26.024036 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d7330-2c47-4534-aca9-5861347597d7 admin admin] PATH_INFO: `/v3/projects/49f42da3393441458d857e5f8d642df0` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:26.035970 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8d7330-2c47-4534-aca9-5861347597d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49f42da3393441458d857e5f8d642df0)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:26.037920 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8d7330-2c47-4534-aca9-5861347597d7 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:26.039930 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b728f9a-bc45-4851-8faa-e0b8ffa528ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:26.040652 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b728f9a-bc45-4851-8faa-e0b8ffa528ab admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:26.040932 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b728f9a-bc45-4851-8faa-e0b8ffa528ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:26.041204 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b728f9a-bc45-4851-8faa-e0b8ffa528ab admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:26.041886 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b728f9a-bc45-4851-8faa-e0b8ffa528ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:26.044090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b728f9a-bc45-4851-8faa-e0b8ffa528ab admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:26.065118 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1337/2678] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:28:25 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:28:26.066074 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b728f9a-bc45-4851-8faa-e0b8ffa528ab admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:26.068084 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1b728f9a-bc45-4851-8faa-e0b8ffa528ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:26.087517 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-fe8d7330-2c47-4534-aca9-5861347597d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49f42da3393441458d857e5f8d642df0', 'request_id': 'req-fe8d7330-2c47-4534-aca9-5861347597d7'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:26.094671 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-fe8d7330-2c47-4534-aca9-5861347597d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49f42da3393441458d857e5f8d642df0', 'request_id': 'req-fe8d7330-2c47-4534-aca9-5861347597d7'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:26.103761 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-fe8d7330-2c47-4534-aca9-5861347597d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49f42da3393441458d857e5f8d642df0', 'request_id': 'req-fe8d7330-2c47-4534-aca9-5861347597d7'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:26.114442 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-fe8d7330-2c47-4534-aca9-5861347597d7 admin admin] The token cache is being invalidate because project 49f42da3393441458d857e5f8d642df0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:28:26.114738 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-fe8d7330-2c47-4534-aca9-5861347597d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:26.115389 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-fe8d7330-2c47-4534-aca9-5861347597d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:26.117027 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1342/2679] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:28:26 2022] DELETE /identity/v3/projects/49f42da3393441458d857e5f8d642df0 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:26.117774 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d7330-2c47-4534-aca9-5861347597d7 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:26.124611 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-fe8d7330-2c47-4534-aca9-5861347597d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:26.138324 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b728f9a-bc45-4851-8faa-e0b8ffa528ab admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:26.167878 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:26.205851 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adadbc58-31ef-4c08-93ce-2a6b91cee994 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:26.206514 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadbc58-31ef-4c08-93ce-2a6b91cee994 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:26.206795 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadbc58-31ef-4c08-93ce-2a6b91cee994 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:26.207237 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadbc58-31ef-4c08-93ce-2a6b91cee994 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:26.209295 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-adadbc58-31ef-4c08-93ce-2a6b91cee994 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:26.234819 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d7330-2c47-4534-aca9-5861347597d7 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:26.244061 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ac8bf8-3154-40ce-86e2-3f672cb03384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:26.244889 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac8bf8-3154-40ce-86e2-3f672cb03384 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:26.245725 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac8bf8-3154-40ce-86e2-3f672cb03384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:26.246021 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac8bf8-3154-40ce-86e2-3f672cb03384 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:26.248030 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ac8bf8-3154-40ce-86e2-3f672cb03384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:26.252196 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ac8bf8-3154-40ce-86e2-3f672cb03384 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:26.279587 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1343/2680] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:28:26.281664 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ac8bf8-3154-40ce-86e2-3f672cb03384 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:26.291806 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adadbc58-31ef-4c08-93ce-2a6b91cee994 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:26.293943 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adadbc58-31ef-4c08-93ce-2a6b91cee994 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:26.298883 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ac8bf8-3154-40ce-86e2-3f672cb03384 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:26.305266 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebc144c-f783-444f-9ec3-ca5d1b4ffebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:26.305724 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1338/2681] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:28:26.305958 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc144c-f783-444f-9ec3-ca5d1b4ffebc admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:26.306248 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc144c-f783-444f-9ec3-ca5d1b4ffebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:26.306531 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebc144c-f783-444f-9ec3-ca5d1b4ffebc admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:26.307279 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebc144c-f783-444f-9ec3-ca5d1b4ffebc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:26.309688 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebc144c-f783-444f-9ec3-ca5d1b4ffebc admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:26.323583 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1344/2682] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:28:26.326155 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adadbc58-31ef-4c08-93ce-2a6b91cee994 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:26.337287 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adadbc58-31ef-4c08-93ce-2a6b91cee994 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:26.343130 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e09192-4de7-4564-9df3-26452f6c2a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:26.345046 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e09192-4de7-4564-9df3-26452f6c2a3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:26.345333 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e09192-4de7-4564-9df3-26452f6c2a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:26.345715 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e09192-4de7-4564-9df3-26452f6c2a3f admin admin] PATH_INFO: `/v3/projects/f7316f34b0e24550a3faf7718061ad80/users/0ff7ba732eb74b30917dfe12b163c12a/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:26.367777 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e09192-4de7-4564-9df3-26452f6c2a3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7316f34b0e24550a3faf7718061ad80, user_id=0ff7ba732eb74b30917dfe12b163c12a, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:26.370972 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e09192-4de7-4564-9df3-26452f6c2a3f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:26.375508 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:26.392408 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1339/2683] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:28:26 2022] PUT /identity/v3/projects/f7316f34b0e24550a3faf7718061ad80/users/0ff7ba732eb74b30917dfe12b163c12a/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:26.403676 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed306a10-a895-4de0-903c-6941222134a7 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:26.404173 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed306a10-a895-4de0-903c-6941222134a7 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:26.404312 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed306a10-a895-4de0-903c-6941222134a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:26.437050 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-ed306a10-a895-4de0-903c-6941222134a7 None None] MFA Rules not processed for user `0ff7ba732eb74b30917dfe12b163c12a`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:26.438283 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-ed306a10-a895-4de0-903c-6941222134a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:26.522205 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1345/2684] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:26.528417 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e09192-4de7-4564-9df3-26452f6c2a3f admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:26.624470 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e09192-4de7-4564-9df3-26452f6c2a3f admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:26.631862 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb42b1b-4ed1-49d5-ba5f-e92675e17ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:26.632673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb42b1b-4ed1-49d5-ba5f-e92675e17ac3 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:26.633110 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb42b1b-4ed1-49d5-ba5f-e92675e17ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:26.633402 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb42b1b-4ed1-49d5-ba5f-e92675e17ac3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:26.634185 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb42b1b-4ed1-49d5-ba5f-e92675e17ac3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:26.636720 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb42b1b-4ed1-49d5-ba5f-e92675e17ac3 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:26.674637 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1340/2685] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:28:26 2022] POST /identity/v3/projects => generated 385 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:28:26.678536 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed306a10-a895-4de0-903c-6941222134a7 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:26.697577 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed306a10-a895-4de0-903c-6941222134a7 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:26.796931 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b118d-3473-4dce-a4ad-f169abc75c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:26.797680 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b118d-3473-4dce-a4ad-f169abc75c22 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:26.797961 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b118d-3473-4dce-a4ad-f169abc75c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:26.798236 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b118d-3473-4dce-a4ad-f169abc75c22 admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:26.799138 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b118d-3473-4dce-a4ad-f169abc75c22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:26.803223 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b118d-3473-4dce-a4ad-f169abc75c22 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:26.830555 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1346/2686] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:28:26 2022] POST /identity/v3/users => generated 293 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:28:26.834716 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb42b1b-4ed1-49d5-ba5f-e92675e17ac3 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:26.845789 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb42b1b-4ed1-49d5-ba5f-e92675e17ac3 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:26.851250 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95db18c5-1488-49f1-81a4-1154049cab84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:26.852371 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db18c5-1488-49f1-81a4-1154049cab84 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:26.852371 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db18c5-1488-49f1-81a4-1154049cab84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:26.852766 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db18c5-1488-49f1-81a4-1154049cab84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:26.853651 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95db18c5-1488-49f1-81a4-1154049cab84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:26.855492 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95db18c5-1488-49f1-81a4-1154049cab84 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:26.869812 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1341/2687] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:28:26.874111 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b118d-3473-4dce-a4ad-f169abc75c22 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:26.884745 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b118d-3473-4dce-a4ad-f169abc75c22 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:26.890510 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd549d-d49c-4c14-906d-d5868f35f261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:26.891323 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd549d-d49c-4c14-906d-d5868f35f261 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:26.891744 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd549d-d49c-4c14-906d-d5868f35f261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:26.892695 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd549d-d49c-4c14-906d-d5868f35f261 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:26.892957 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cd549d-d49c-4c14-906d-d5868f35f261 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:26.894707 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cd549d-d49c-4c14-906d-d5868f35f261 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:26.906585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1347/2688] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:28:26.910517 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95db18c5-1488-49f1-81a4-1154049cab84 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:26.922908 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95db18c5-1488-49f1-81a4-1154049cab84 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:26.929721 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66990fd0-d299-4fe1-ab94-d4aa6281727b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:26.930701 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66990fd0-d299-4fe1-ab94-d4aa6281727b admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:26.931069 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66990fd0-d299-4fe1-ab94-d4aa6281727b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:26.931725 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66990fd0-d299-4fe1-ab94-d4aa6281727b admin admin] PATH_INFO: `/v3/projects/9c4545b376b540b4bd9c82dceb1c1d56/users/a113fdc7212f496896934da861a8fbe6/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:26.952080 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66990fd0-d299-4fe1-ab94-d4aa6281727b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c4545b376b540b4bd9c82dceb1c1d56, user_id=a113fdc7212f496896934da861a8fbe6, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:26.954957 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66990fd0-d299-4fe1-ab94-d4aa6281727b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:26.974748 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1342/2689] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:28:26 2022] PUT /identity/v3/projects/9c4545b376b540b4bd9c82dceb1c1d56/users/a113fdc7212f496896934da861a8fbe6/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:28:26.990814 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99634a47-1acc-4197-95b5-09126f15a045 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:26.991109 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99634a47-1acc-4197-95b5-09126f15a045 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:26.991394 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99634a47-1acc-4197-95b5-09126f15a045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:27.019904 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-99634a47-1acc-4197-95b5-09126f15a045 None None] MFA Rules not processed for user `a113fdc7212f496896934da861a8fbe6`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:27.021277 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-99634a47-1acc-4197-95b5-09126f15a045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:27.025561 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:27.109210 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1348/2690] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:28:27.113516 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66990fd0-d299-4fe1-ab94-d4aa6281727b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:27.187267 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66990fd0-d299-4fe1-ab94-d4aa6281727b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:27.192322 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce02a32-c059-4f49-8803-9a5b005925d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:27.193046 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce02a32-c059-4f49-8803-9a5b005925d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:27.193399 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce02a32-c059-4f49-8803-9a5b005925d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:27.193886 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce02a32-c059-4f49-8803-9a5b005925d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:27.194760 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce02a32-c059-4f49-8803-9a5b005925d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:27.196777 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce02a32-c059-4f49-8803-9a5b005925d1 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:27.228390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1343/2691] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:28:27 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:28:27.232689 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99634a47-1acc-4197-95b5-09126f15a045 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:27.246304 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99634a47-1acc-4197-95b5-09126f15a045 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:27.320368 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f0a5a-50fb-4449-8c49-f1c3a8da227c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:27.321038 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f0a5a-50fb-4449-8c49-f1c3a8da227c admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:27.321350 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f0a5a-50fb-4449-8c49-f1c3a8da227c admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:27.321627 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f0a5a-50fb-4449-8c49-f1c3a8da227c admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:27.322316 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643f0a5a-50fb-4449-8c49-f1c3a8da227c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:27.325139 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643f0a5a-50fb-4449-8c49-f1c3a8da227c admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:27.347389 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1349/2692] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:28:27 2022] POST /identity/v3/users => generated 300 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:28:27.352032 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce02a32-c059-4f49-8803-9a5b005925d1 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:27.365499 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce02a32-c059-4f49-8803-9a5b005925d1 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:27.370686 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cd096d-401d-4934-b562-27bbd5b1fe59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:27.371916 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd096d-401d-4934-b562-27bbd5b1fe59 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:27.372201 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd096d-401d-4934-b562-27bbd5b1fe59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:27.372479 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd096d-401d-4934-b562-27bbd5b1fe59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:27.373599 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cd096d-401d-4934-b562-27bbd5b1fe59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:27.374820 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cd096d-401d-4934-b562-27bbd5b1fe59 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:27.386343 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1344/2693] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:28:27.391124 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f0a5a-50fb-4449-8c49-f1c3a8da227c admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:27.401511 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f0a5a-50fb-4449-8c49-f1c3a8da227c admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:27.406566 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ded082-e9de-4238-b53a-4637ad04aecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:27.407522 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ded082-e9de-4238-b53a-4637ad04aecf admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:27.407840 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ded082-e9de-4238-b53a-4637ad04aecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:27.408272 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ded082-e9de-4238-b53a-4637ad04aecf admin admin] PATH_INFO: `/v3/projects/48fd1a09060349a5a971bffa83a283b3/users/de6c1b5c8f7c427c9d726b26222386cb/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:27.425139 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ded082-e9de-4238-b53a-4637ad04aecf admin admin] RBAC: Authorizing `identity:create_grant(project_id=48fd1a09060349a5a971bffa83a283b3, user_id=de6c1b5c8f7c427c9d726b26222386cb, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:27.427871 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ded082-e9de-4238-b53a-4637ad04aecf admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:27.445293 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1350/2694] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:28:27 2022] PUT /identity/v3/projects/48fd1a09060349a5a971bffa83a283b3/users/de6c1b5c8f7c427c9d726b26222386cb/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:27.449423 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cd096d-401d-4934-b562-27bbd5b1fe59 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:27.461464 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cd096d-401d-4934-b562-27bbd5b1fe59 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:27.476214 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:27.531587 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca9ed0-218d-4aa9-89bd-6f16be7ab777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:27.532726 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca9ed0-218d-4aa9-89bd-6f16be7ab777 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:27.532726 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca9ed0-218d-4aa9-89bd-6f16be7ab777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:27.532936 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca9ed0-218d-4aa9-89bd-6f16be7ab777 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:27.533551 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca9ed0-218d-4aa9-89bd-6f16be7ab777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:27.535228 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca9ed0-218d-4aa9-89bd-6f16be7ab777 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:27.549010 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1345/2695] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:27 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:28:27.552909 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ded082-e9de-4238-b53a-4637ad04aecf admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:27.564751 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ded082-e9de-4238-b53a-4637ad04aecf admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:27.569794 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312dbc0-9ab2-4e7f-9669-c0892a51baac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:27.570713 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312dbc0-9ab2-4e7f-9669-c0892a51baac admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:27.571155 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312dbc0-9ab2-4e7f-9669-c0892a51baac admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:27.571436 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312dbc0-9ab2-4e7f-9669-c0892a51baac admin admin] PATH_INFO: `/v3/projects/48fd1a09060349a5a971bffa83a283b3/users/de6c1b5c8f7c427c9d726b26222386cb/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:27.573592 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312dbc0-9ab2-4e7f-9669-c0892a51baac admin admin] RBAC: Authorizing `identity:create_grant(project_id=48fd1a09060349a5a971bffa83a283b3, user_id=de6c1b5c8f7c427c9d726b26222386cb, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:27.576957 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312dbc0-9ab2-4e7f-9669-c0892a51baac admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:27.588780 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1351/2696] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:28:27 2022] PUT /identity/v3/projects/48fd1a09060349a5a971bffa83a283b3/users/de6c1b5c8f7c427c9d726b26222386cb/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:27.599524 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa87b5-3ae6-4bd4-a049-b3c26abf1d01 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:27.599779 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa87b5-3ae6-4bd4-a049-b3c26abf1d01 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:27.600008 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa87b5-3ae6-4bd4-a049-b3c26abf1d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:27.631780 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-ceaa87b5-3ae6-4bd4-a049-b3c26abf1d01 None None] MFA Rules not processed for user `de6c1b5c8f7c427c9d726b26222386cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:27.633168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-ceaa87b5-3ae6-4bd4-a049-b3c26abf1d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:27.723789 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1346/2697] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:27 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:27.735664 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312dbc0-9ab2-4e7f-9669-c0892a51baac admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:27.737433 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-c312dbc0-9ab2-4e7f-9669-c0892a51baac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:27.818765 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312dbc0-9ab2-4e7f-9669-c0892a51baac admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:27.823811 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0bd694-0e33-4f4e-80df-a935ca8e31e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:27.825107 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bd694-0e33-4f4e-80df-a935ca8e31e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:27.825455 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bd694-0e33-4f4e-80df-a935ca8e31e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:27.825824 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bd694-0e33-4f4e-80df-a935ca8e31e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:27.839061 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0bd694-0e33-4f4e-80df-a935ca8e31e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:27.841508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0bd694-0e33-4f4e-80df-a935ca8e31e5 service cinder] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:27.848015 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:27.855723 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1352/2698] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:27 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:28:28.454262 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa87b5-3ae6-4bd4-a049-b3c26abf1d01 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:28.465600 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa87b5-3ae6-4bd4-a049-b3c26abf1d01 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:28.469588 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6151a-a964-4c6a-8ca2-5900bb45b59e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:28.470355 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6151a-a964-4c6a-8ca2-5900bb45b59e service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:28.470710 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6151a-a964-4c6a-8ca2-5900bb45b59e service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:28.471068 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6151a-a964-4c6a-8ca2-5900bb45b59e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:28.538350 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6151a-a964-4c6a-8ca2-5900bb45b59e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:28.540284 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6151a-a964-4c6a-8ca2-5900bb45b59e service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:28.553107 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1347/2699] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:28:30.662644 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c990f66-b15d-43df-8de4-5449c21567f3 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:30.663188 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c990f66-b15d-43df-8de4-5449c21567f3 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:30.663188 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c990f66-b15d-43df-8de4-5449c21567f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:30.699873 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-6c990f66-b15d-43df-8de4-5449c21567f3 None None] MFA Rules not processed for user `e292de524b9c405dae3d520b60136155`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:30.700899 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-6c990f66-b15d-43df-8de4-5449c21567f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:30.833353 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1353/2700] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:30 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:30.838634 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6151a-a964-4c6a-8ca2-5900bb45b59e service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:30.840762 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e5d6151a-a964-4c6a-8ca2-5900bb45b59e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:30.911767 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:30.949030 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6151a-a964-4c6a-8ca2-5900bb45b59e service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:30.957525 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f77c679-3717-462b-bb7c-80f703c8ccf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:30.958632 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77c679-3717-462b-bb7c-80f703c8ccf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:30.958927 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77c679-3717-462b-bb7c-80f703c8ccf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:30.959210 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77c679-3717-462b-bb7c-80f703c8ccf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:30.959969 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f77c679-3717-462b-bb7c-80f703c8ccf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:30.962569 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f77c679-3717-462b-bb7c-80f703c8ccf3 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:31.005141 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1348/2701] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:28:30 2022] POST /identity/v3/projects => generated 373 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:28:31.008506 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c990f66-b15d-43df-8de4-5449c21567f3 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:31.026177 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c990f66-b15d-43df-8de4-5449c21567f3 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:31.115785 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c8733c-cc5e-40df-a060-59663796e107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:31.116956 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c8733c-cc5e-40df-a060-59663796e107 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:31.117106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c8733c-cc5e-40df-a060-59663796e107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:31.117443 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c8733c-cc5e-40df-a060-59663796e107 admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:31.118250 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c8733c-cc5e-40df-a060-59663796e107 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:31.120812 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c8733c-cc5e-40df-a060-59663796e107 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:31.148691 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1354/2702] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:28:31 2022] POST /identity/v3/users => generated 293 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:28:31.152371 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f77c679-3717-462b-bb7c-80f703c8ccf3 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:31.166606 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f77c679-3717-462b-bb7c-80f703c8ccf3 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:31.174950 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ff0b7-251a-4854-bfbf-9aeff3309c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:31.175762 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ff0b7-251a-4854-bfbf-9aeff3309c6e admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:31.176147 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ff0b7-251a-4854-bfbf-9aeff3309c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:31.176497 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ff0b7-251a-4854-bfbf-9aeff3309c6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:31.177321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ff0b7-251a-4854-bfbf-9aeff3309c6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:31.179207 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ff0b7-251a-4854-bfbf-9aeff3309c6e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:31.196231 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1349/2703] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:31 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:28:31.199161 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c8733c-cc5e-40df-a060-59663796e107 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:31.213294 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c8733c-cc5e-40df-a060-59663796e107 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:31.218787 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddfdc3d-062c-4cf6-8c6b-5be3860d3cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:31.219718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddfdc3d-062c-4cf6-8c6b-5be3860d3cc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:31.220005 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddfdc3d-062c-4cf6-8c6b-5be3860d3cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:31.220405 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddfdc3d-062c-4cf6-8c6b-5be3860d3cc8 admin admin] PATH_INFO: `/v3/projects/c13ca5d8f19349e19431c40f3bae5893/users/9440975fa183447bb404c1ea01117c61/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:31.241514 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddfdc3d-062c-4cf6-8c6b-5be3860d3cc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c13ca5d8f19349e19431c40f3bae5893, user_id=9440975fa183447bb404c1ea01117c61, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:31.245346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddfdc3d-062c-4cf6-8c6b-5be3860d3cc8 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:31.259548 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1355/2704] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:28:31 2022] PUT /identity/v3/projects/c13ca5d8f19349e19431c40f3bae5893/users/9440975fa183447bb404c1ea01117c61/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:31.264352 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ff0b7-251a-4854-bfbf-9aeff3309c6e admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:31.275142 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ff0b7-251a-4854-bfbf-9aeff3309c6e admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:31.337244 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf516c4-c85f-4576-9208-b7bcc4ef9ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:31.338057 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf516c4-c85f-4576-9208-b7bcc4ef9ec9 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:31.338542 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf516c4-c85f-4576-9208-b7bcc4ef9ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:31.338828 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf516c4-c85f-4576-9208-b7bcc4ef9ec9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:31.339586 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf516c4-c85f-4576-9208-b7bcc4ef9ec9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:31.341284 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf516c4-c85f-4576-9208-b7bcc4ef9ec9 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:31.352028 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1350/2705] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:31 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:28:31.360495 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddfdc3d-062c-4cf6-8c6b-5be3860d3cc8 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:31.375036 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddfdc3d-062c-4cf6-8c6b-5be3860d3cc8 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:31.380584 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ffbdf-7b24-4662-bb94-7cfc61e93762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:31.381487 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ffbdf-7b24-4662-bb94-7cfc61e93762 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:31.381772 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ffbdf-7b24-4662-bb94-7cfc61e93762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:31.382112 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ffbdf-7b24-4662-bb94-7cfc61e93762 admin admin] PATH_INFO: `/v3/projects/c13ca5d8f19349e19431c40f3bae5893/users/9440975fa183447bb404c1ea01117c61/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:31.384362 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522ffbdf-7b24-4662-bb94-7cfc61e93762 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c13ca5d8f19349e19431c40f3bae5893, user_id=9440975fa183447bb404c1ea01117c61, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:31.387664 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522ffbdf-7b24-4662-bb94-7cfc61e93762 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:31.398925 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1356/2706] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:28:31 2022] PUT /identity/v3/projects/c13ca5d8f19349e19431c40f3bae5893/users/9440975fa183447bb404c1ea01117c61/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:31.642038 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf516c4-c85f-4576-9208-b7bcc4ef9ec9 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:31.644222 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5bf516c4-c85f-4576-9208-b7bcc4ef9ec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:31.731577 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf516c4-c85f-4576-9208-b7bcc4ef9ec9 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:31.797601 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:31.816944 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59106d63-5f6c-45cd-8b2a-c52193c56a84 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64824ba84a024930bbc177f8a8d9a046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:31.817761 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59106d63-5f6c-45cd-8b2a-c52193c56a84 service placement] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:31.818072 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59106d63-5f6c-45cd-8b2a-c52193c56a84 service placement] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:31.818351 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59106d63-5f6c-45cd-8b2a-c52193c56a84 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:31.820292 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-59106d63-5f6c-45cd-8b2a-c52193c56a84 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:31.833151 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59106d63-5f6c-45cd-8b2a-c52193c56a84 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:31.834799 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59106d63-5f6c-45cd-8b2a-c52193c56a84 service placement] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:31.846013 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1351/2707] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:28:35.256017 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ffbdf-7b24-4662-bb94-7cfc61e93762 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:35.429293 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:35.447043 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ffbdf-7b24-4662-bb94-7cfc61e93762 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:35.461620 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff138b-c561-43c5-ad9e-49be9e20ece7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:35.462486 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff138b-c561-43c5-ad9e-49be9e20ece7 service cinder] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:35.462860 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff138b-c561-43c5-ad9e-49be9e20ece7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:35.463208 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff138b-c561-43c5-ad9e-49be9e20ece7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:35.620776 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff138b-c561-43c5-ad9e-49be9e20ece7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:35.623938 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff138b-c561-43c5-ad9e-49be9e20ece7 service cinder] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:35.656518 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1357/2708] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:28:35.869759 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59106d63-5f6c-45cd-8b2a-c52193c56a84 service placement] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:35.871897 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-59106d63-5f6c-45cd-8b2a-c52193c56a84 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:36.077037 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59106d63-5f6c-45cd-8b2a-c52193c56a84 service placement] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:36.089533 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236734be-a76a-4d9d-9c09-f2c447238c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:36.093144 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236734be-a76a-4d9d-9c09-f2c447238c16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:36.093430 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236734be-a76a-4d9d-9c09-f2c447238c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:36.093707 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236734be-a76a-4d9d-9c09-f2c447238c16 admin admin] PATH_INFO: `/v3/users/350a76fe0de54add881ff71750fcbaf8` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:36.099152 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236734be-a76a-4d9d-9c09-f2c447238c16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=350a76fe0de54add881ff71750fcbaf8)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:36.102533 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236734be-a76a-4d9d-9c09-f2c447238c16 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:36.205887 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-236734be-a76a-4d9d-9c09-f2c447238c16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '350a76fe0de54add881ff71750fcbaf8'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:36.206894 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-236734be-a76a-4d9d-9c09-f2c447238c16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '350a76fe0de54add881ff71750fcbaf8'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:36.228855 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-236734be-a76a-4d9d-9c09-f2c447238c16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '350a76fe0de54add881ff71750fcbaf8'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:36.247818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-236734be-a76a-4d9d-9c09-f2c447238c16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '350a76fe0de54add881ff71750fcbaf8'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:36.259929 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-236734be-a76a-4d9d-9c09-f2c447238c16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '350a76fe0de54add881ff71750fcbaf8'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:36.283440 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1352/2709] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:28:35 2022] DELETE /identity/v3/users/350a76fe0de54add881ff71750fcbaf8 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:36.290247 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99294678-b460-4fd5-9f0c-d6cc9e59de15 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:36.292297 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99294678-b460-4fd5-9f0c-d6cc9e59de15 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:36.293602 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99294678-b460-4fd5-9f0c-d6cc9e59de15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:36.336349 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-99294678-b460-4fd5-9f0c-d6cc9e59de15 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:36.344431 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-99294678-b460-4fd5-9f0c-d6cc9e59de15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:36.469932 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1358/2710] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:36.484131 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236734be-a76a-4d9d-9c09-f2c447238c16 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:36.486073 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-236734be-a76a-4d9d-9c09-f2c447238c16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:36.634994 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236734be-a76a-4d9d-9c09-f2c447238c16 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:36.649008 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b53d20-3067-4c37-bf98-67c4e8e93c09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:36.649008 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b53d20-3067-4c37-bf98-67c4e8e93c09 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:36.649008 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b53d20-3067-4c37-bf98-67c4e8e93c09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:36.649008 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b53d20-3067-4c37-bf98-67c4e8e93c09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:36.661790 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b53d20-3067-4c37-bf98-67c4e8e93c09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:36.663285 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b53d20-3067-4c37-bf98-67c4e8e93c09 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:36.675602 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1353/2711] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:28:37.136792 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99294678-b460-4fd5-9f0c-d6cc9e59de15 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:37.138925 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-99294678-b460-4fd5-9f0c-d6cc9e59de15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:37.166701 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99294678-b460-4fd5-9f0c-d6cc9e59de15 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:37.178736 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e9222-7224-4792-9d7c-c7f6a80df0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:37.180415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e9222-7224-4792-9d7c-c7f6a80df0c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:37.180791 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e9222-7224-4792-9d7c-c7f6a80df0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:37.181220 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e9222-7224-4792-9d7c-c7f6a80df0c3 admin admin] PATH_INFO: `/v3/projects/e900e3c84f394cdd9653999f94084fda` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:37.182688 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6e9222-7224-4792-9d7c-c7f6a80df0c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e900e3c84f394cdd9653999f94084fda)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:37.185946 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6e9222-7224-4792-9d7c-c7f6a80df0c3 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:37.270911 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-5e6e9222-7224-4792-9d7c-c7f6a80df0c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e900e3c84f394cdd9653999f94084fda', 'request_id': 'req-5e6e9222-7224-4792-9d7c-c7f6a80df0c3'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:37.279556 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-5e6e9222-7224-4792-9d7c-c7f6a80df0c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e900e3c84f394cdd9653999f94084fda', 'request_id': 'req-5e6e9222-7224-4792-9d7c-c7f6a80df0c3'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:37.326337 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-5e6e9222-7224-4792-9d7c-c7f6a80df0c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e900e3c84f394cdd9653999f94084fda', 'request_id': 'req-5e6e9222-7224-4792-9d7c-c7f6a80df0c3'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:37.343798 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-5e6e9222-7224-4792-9d7c-c7f6a80df0c3 admin admin] The token cache is being invalidate because project e900e3c84f394cdd9653999f94084fda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:28:37.344155 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-5e6e9222-7224-4792-9d7c-c7f6a80df0c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:37.344887 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-5e6e9222-7224-4792-9d7c-c7f6a80df0c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:37.346534 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1359/2712] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:28:37 2022] DELETE /identity/v3/projects/e900e3c84f394cdd9653999f94084fda => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:37.350455 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b53d20-3067-4c37-bf98-67c4e8e93c09 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:37.366031 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:37.378864 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b53d20-3067-4c37-bf98-67c4e8e93c09 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:37.425434 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a6ba46-5ef5-4260-b04b-669db17119d4 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:37.425840 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a6ba46-5ef5-4260-b04b-669db17119d4 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:37.426193 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a6ba46-5ef5-4260-b04b-669db17119d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:37.463505 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d92943-6df7-41ea-b5cb-1b8a4cce9295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:37.464424 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d92943-6df7-41ea-b5cb-1b8a4cce9295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:37.464707 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d92943-6df7-41ea-b5cb-1b8a4cce9295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:37.465544 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d92943-6df7-41ea-b5cb-1b8a4cce9295 admin admin] PATH_INFO: `/v3/users/b8f466a8fc23477ea4c98a09631c71e7` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:37.466766 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d92943-6df7-41ea-b5cb-1b8a4cce9295 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8f466a8fc23477ea4c98a09631c71e7)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:37.468052 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-77a6ba46-5ef5-4260-b04b-669db17119d4 None None] MFA Rules not processed for user `9440975fa183447bb404c1ea01117c61`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:37.468732 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d92943-6df7-41ea-b5cb-1b8a4cce9295 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:37.469907 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-77a6ba46-5ef5-4260-b04b-669db17119d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:37.480392 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:37.529556 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-68d92943-6df7-41ea-b5cb-1b8a4cce9295 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8f466a8fc23477ea4c98a09631c71e7'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:37.530278 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-68d92943-6df7-41ea-b5cb-1b8a4cce9295 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8f466a8fc23477ea4c98a09631c71e7'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:37.546330 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-68d92943-6df7-41ea-b5cb-1b8a4cce9295 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8f466a8fc23477ea4c98a09631c71e7'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:37.554864 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-68d92943-6df7-41ea-b5cb-1b8a4cce9295 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8f466a8fc23477ea4c98a09631c71e7'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:37.564261 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-68d92943-6df7-41ea-b5cb-1b8a4cce9295 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8f466a8fc23477ea4c98a09631c71e7'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:37.572119 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1354/2713] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:28:37 2022] DELETE /identity/v3/users/b8f466a8fc23477ea4c98a09631c71e7 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:37.576629 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1360/2714] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:37 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:28:37.595861 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d92943-6df7-41ea-b5cb-1b8a4cce9295 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:37.597383 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-68d92943-6df7-41ea-b5cb-1b8a4cce9295 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:37.687159 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d92943-6df7-41ea-b5cb-1b8a4cce9295 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:37.694790 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b39128-ee6e-48ff-ae1a-382836053ad6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:37.696145 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b39128-ee6e-48ff-ae1a-382836053ad6 service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:37.696601 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b39128-ee6e-48ff-ae1a-382836053ad6 service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:37.697055 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b39128-ee6e-48ff-ae1a-382836053ad6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:37.699888 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-52b39128-ee6e-48ff-ae1a-382836053ad6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:37.724116 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:37.790541 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b39128-ee6e-48ff-ae1a-382836053ad6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:37.792359 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b39128-ee6e-48ff-ae1a-382836053ad6 service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:37.804240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1355/2715] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:37 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:28:37.929671 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a6ba46-5ef5-4260-b04b-669db17119d4 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:37.931103 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-77a6ba46-5ef5-4260-b04b-669db17119d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:37.955743 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a6ba46-5ef5-4260-b04b-669db17119d4 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:38.031332 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5921bec-6f70-4812-ac79-edab615f204e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:38.032291 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5921bec-6f70-4812-ac79-edab615f204e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:38.032598 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5921bec-6f70-4812-ac79-edab615f204e admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:38.033058 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5921bec-6f70-4812-ac79-edab615f204e admin admin] PATH_INFO: `/v3/projects/83931a19de3c4a6ab3af4824a800b85b` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:38.034291 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5921bec-6f70-4812-ac79-edab615f204e admin admin] RBAC: Authorizing `identity:delete_project(project_id=83931a19de3c4a6ab3af4824a800b85b)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:38.036531 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5921bec-6f70-4812-ac79-edab615f204e admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:38.099787 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-b5921bec-6f70-4812-ac79-edab615f204e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83931a19de3c4a6ab3af4824a800b85b', 'request_id': 'req-b5921bec-6f70-4812-ac79-edab615f204e'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:38.107236 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-b5921bec-6f70-4812-ac79-edab615f204e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83931a19de3c4a6ab3af4824a800b85b', 'request_id': 'req-b5921bec-6f70-4812-ac79-edab615f204e'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:38.123287 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-b5921bec-6f70-4812-ac79-edab615f204e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83931a19de3c4a6ab3af4824a800b85b', 'request_id': 'req-b5921bec-6f70-4812-ac79-edab615f204e'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:38.138393 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-b5921bec-6f70-4812-ac79-edab615f204e admin admin] The token cache is being invalidate because project 83931a19de3c4a6ab3af4824a800b85b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:28:38.138771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-b5921bec-6f70-4812-ac79-edab615f204e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:38.139875 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-b5921bec-6f70-4812-ac79-edab615f204e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:38.141726 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1361/2716] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:28:37 2022] DELETE /identity/v3/projects/83931a19de3c4a6ab3af4824a800b85b => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:38.147110 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346df8c-cb4a-49dc-b86e-9c92e5149875 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:38.147406 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346df8c-cb4a-49dc-b86e-9c92e5149875 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:38.147725 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6346df8c-cb4a-49dc-b86e-9c92e5149875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:38.173823 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-6346df8c-cb4a-49dc-b86e-9c92e5149875 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:38.175180 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6346df8c-cb4a-49dc-b86e-9c92e5149875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:38.271756 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1356/2717] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:38.276283 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078fb34a-eb33-46ff-83b6-20d37988c4db None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:38.276844 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078fb34a-eb33-46ff-83b6-20d37988c4db None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:38.277307 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078fb34a-eb33-46ff-83b6-20d37988c4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:38.305843 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-078fb34a-eb33-46ff-83b6-20d37988c4db None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:38.307020 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-078fb34a-eb33-46ff-83b6-20d37988c4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:38.325404 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1362/2718] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:28:38.330785 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346df8c-cb4a-49dc-b86e-9c92e5149875 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:38.345873 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6346df8c-cb4a-49dc-b86e-9c92e5149875 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:38.352028 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec6ba7d-1a60-4d09-89ba-753645fb601f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:38.352947 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec6ba7d-1a60-4d09-89ba-753645fb601f admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:38.353378 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec6ba7d-1a60-4d09-89ba-753645fb601f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:38.353587 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec6ba7d-1a60-4d09-89ba-753645fb601f admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:38.360463 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec6ba7d-1a60-4d09-89ba-753645fb601f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:28:38.360463 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec6ba7d-1a60-4d09-89ba-753645fb601f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:38.360463 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec6ba7d-1a60-4d09-89ba-753645fb601f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:38.376776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1357/2719] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:28:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:28:38.381406 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078fb34a-eb33-46ff-83b6-20d37988c4db None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:38.391296 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078fb34a-eb33-46ff-83b6-20d37988c4db None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:38.398392 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308139d3-931e-4ff0-af4c-ab027baa2d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:38.399367 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308139d3-931e-4ff0-af4c-ab027baa2d9a admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:38.399845 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308139d3-931e-4ff0-af4c-ab027baa2d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:38.400147 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308139d3-931e-4ff0-af4c-ab027baa2d9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:38.401392 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308139d3-931e-4ff0-af4c-ab027baa2d9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:38.404212 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308139d3-931e-4ff0-af4c-ab027baa2d9a admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:38.437636 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1363/2720] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:28:38 2022] POST /identity/v3/projects => generated 393 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:28:38.442160 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec6ba7d-1a60-4d09-89ba-753645fb601f admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:38.454632 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec6ba7d-1a60-4d09-89ba-753645fb601f admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:38.540346 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f0e879-d0d0-4355-8d52-a924db95f093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:38.541525 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f0e879-d0d0-4355-8d52-a924db95f093 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:38.541525 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f0e879-d0d0-4355-8d52-a924db95f093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:38.541776 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f0e879-d0d0-4355-8d52-a924db95f093 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:38.545089 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f0e879-d0d0-4355-8d52-a924db95f093 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:38.547509 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f0e879-d0d0-4355-8d52-a924db95f093 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:38.580104 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1358/2721] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:28:38 2022] POST /identity/v3/users => generated 297 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:28:38.582135 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308139d3-931e-4ff0-af4c-ab027baa2d9a admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:38.598820 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308139d3-931e-4ff0-af4c-ab027baa2d9a admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:38.606633 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd48761-1ff2-440e-b113-b2dfa629c516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:38.607511 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd48761-1ff2-440e-b113-b2dfa629c516 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:38.608003 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd48761-1ff2-440e-b113-b2dfa629c516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:38.608442 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd48761-1ff2-440e-b113-b2dfa629c516 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:38.608832 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f0e879-d0d0-4355-8d52-a924db95f093 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:38.610114 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd48761-1ff2-440e-b113-b2dfa629c516 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:38.610571 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-28f0e879-d0d0-4355-8d52-a924db95f093 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:38.612173 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd48761-1ff2-440e-b113-b2dfa629c516 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:38.614075 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:38.627769 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f0e879-d0d0-4355-8d52-a924db95f093 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:38.630820 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1364/2722] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:38 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:28:38.632067 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd48761-1ff2-440e-b113-b2dfa629c516 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:38.633537 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f008978-cb37-4369-9dec-396c9ac78310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:38.634945 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f008978-cb37-4369-9dec-396c9ac78310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:38.634945 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f008978-cb37-4369-9dec-396c9ac78310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:38.634945 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f008978-cb37-4369-9dec-396c9ac78310 admin admin] PATH_INFO: `/v3/users/0ff7ba732eb74b30917dfe12b163c12a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:38.636217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f008978-cb37-4369-9dec-396c9ac78310 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ff7ba732eb74b30917dfe12b163c12a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:38.638535 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f008978-cb37-4369-9dec-396c9ac78310 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:38.644776 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd48761-1ff2-440e-b113-b2dfa629c516 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:38.650760 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebafc9a-e190-44f0-bcef-b322bccaba27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:38.651436 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebafc9a-e190-44f0-bcef-b322bccaba27 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:38.651820 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebafc9a-e190-44f0-bcef-b322bccaba27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:38.652803 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebafc9a-e190-44f0-bcef-b322bccaba27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:38.653601 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebafc9a-e190-44f0-bcef-b322bccaba27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:38.657158 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebafc9a-e190-44f0-bcef-b322bccaba27 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:38.675671 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1365/2723] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:38 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:28:38.687851 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:38.690553 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebafc9a-e190-44f0-bcef-b322bccaba27 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:38.715597 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4f008978-cb37-4369-9dec-396c9ac78310 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ff7ba732eb74b30917dfe12b163c12a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:38.715597 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4f008978-cb37-4369-9dec-396c9ac78310 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ff7ba732eb74b30917dfe12b163c12a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:38.717278 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebafc9a-e190-44f0-bcef-b322bccaba27 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:38.722567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd940a0-9b5c-4c2a-ab46-ab21a0decb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:38.723474 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd940a0-9b5c-4c2a-ab46-ab21a0decb06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:38.723678 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd940a0-9b5c-4c2a-ab46-ab21a0decb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:38.723953 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd940a0-9b5c-4c2a-ab46-ab21a0decb06 admin admin] PATH_INFO: `/v3/projects/81a13212db634e53bfe355900e90bd9c/users/b2dbd3c8cbf242bcb5b3a8d1931fcb15/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:38.742804 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd940a0-9b5c-4c2a-ab46-ab21a0decb06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81a13212db634e53bfe355900e90bd9c, user_id=b2dbd3c8cbf242bcb5b3a8d1931fcb15, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:38.745941 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd940a0-9b5c-4c2a-ab46-ab21a0decb06 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:38.750998 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4f008978-cb37-4369-9dec-396c9ac78310 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ff7ba732eb74b30917dfe12b163c12a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:38.767887 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4f008978-cb37-4369-9dec-396c9ac78310 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ff7ba732eb74b30917dfe12b163c12a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:38.777108 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1366/2724] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:28:38 2022] PUT /identity/v3/projects/81a13212db634e53bfe355900e90bd9c/users/b2dbd3c8cbf242bcb5b3a8d1931fcb15/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:38.777909 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-4f008978-cb37-4369-9dec-396c9ac78310 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ff7ba732eb74b30917dfe12b163c12a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:38.789702 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9ada6-9d10-4b20-9ea7-7936f3e840df None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:38.790251 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9ada6-9d10-4b20-9ea7-7936f3e840df None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:38.790699 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9ada6-9d10-4b20-9ea7-7936f3e840df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:38.802137 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1359/2725] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:28:38 2022] DELETE /identity/v3/users/0ff7ba732eb74b30917dfe12b163c12a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:38.806971 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015b6be-47b4-485e-8b68-8a35b0e3234f None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:38.807340 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015b6be-47b4-485e-8b68-8a35b0e3234f None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:38.807778 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015b6be-47b4-485e-8b68-8a35b0e3234f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:38.829931 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-4015b6be-47b4-485e-8b68-8a35b0e3234f None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:38.831020 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4015b6be-47b4-485e-8b68-8a35b0e3234f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:38.833008 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-afa9ada6-9d10-4b20-9ea7-7936f3e840df None None] MFA Rules not processed for user `b2dbd3c8cbf242bcb5b3a8d1931fcb15`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:38.834173 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-afa9ada6-9d10-4b20-9ea7-7936f3e840df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:38.908844 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1367/2726] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:38.913526 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9ada6-9d10-4b20-9ea7-7936f3e840df None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:38.915394 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-afa9ada6-9d10-4b20-9ea7-7936f3e840df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:38.934725 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1360/2727] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:38.946240 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4015b6be-47b4-485e-8b68-8a35b0e3234f None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:38.947710 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4015b6be-47b4-485e-8b68-8a35b0e3234f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:39.017764 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9ada6-9d10-4b20-9ea7-7936f3e840df None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:39.024226 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5749eef1-e91a-4be2-b0a9-4946fb494dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:39.024923 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5749eef1-e91a-4be2-b0a9-4946fb494dd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:39.025234 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5749eef1-e91a-4be2-b0a9-4946fb494dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:39.025466 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5749eef1-e91a-4be2-b0a9-4946fb494dd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:39.026171 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5749eef1-e91a-4be2-b0a9-4946fb494dd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:39.030736 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5749eef1-e91a-4be2-b0a9-4946fb494dd2 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:39.057766 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4015b6be-47b4-485e-8b68-8a35b0e3234f None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:39.062004 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca997c-360f-4205-9326-7f423db40304 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:39.062755 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca997c-360f-4205-9326-7f423db40304 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:39.063045 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca997c-360f-4205-9326-7f423db40304 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:39.063379 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca997c-360f-4205-9326-7f423db40304 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:39.076710 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1368/2728] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:28:38 2022] POST /identity/v3/projects => generated 391 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:28:39.076988 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca997c-360f-4205-9326-7f423db40304 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:39.078196 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5749eef1-e91a-4be2-b0a9-4946fb494dd2 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:39.079035 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca997c-360f-4205-9326-7f423db40304 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:39.091244 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1361/2729] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:28:39.128100 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eab50e-5eb2-4f0f-ab45-850423a3f9f4 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:39.128715 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eab50e-5eb2-4f0f-ab45-850423a3f9f4 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:39.129273 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eab50e-5eb2-4f0f-ab45-850423a3f9f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:39.142333 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:39.155764 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-e7eab50e-5eb2-4f0f-ab45-850423a3f9f4 None None] MFA Rules not processed for user `e292de524b9c405dae3d520b60136155`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:39.157630 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e7eab50e-5eb2-4f0f-ab45-850423a3f9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:39.178257 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5749eef1-e91a-4be2-b0a9-4946fb494dd2 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:39.184262 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afe0dac-ffb6-4f62-ba5c-422a5c6038a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:39.184961 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe0dac-ffb6-4f62-ba5c-422a5c6038a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:39.185248 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe0dac-ffb6-4f62-ba5c-422a5c6038a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:39.185528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe0dac-ffb6-4f62-ba5c-422a5c6038a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:39.186551 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afe0dac-ffb6-4f62-ba5c-422a5c6038a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:39.188761 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afe0dac-ffb6-4f62-ba5c-422a5c6038a1 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:39.218178 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1369/2730] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:28:39 2022] POST /identity/v3/users => generated 302 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:28:39.221030 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afe0dac-ffb6-4f62-ba5c-422a5c6038a1 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:39.222695 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:39.246591 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afe0dac-ffb6-4f62-ba5c-422a5c6038a1 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:39.247908 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1362/2731] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:39 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:39.257920 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ffa07d-71f0-4b78-a88d-2d17c2ec21dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:39.258850 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ffa07d-71f0-4b78-a88d-2d17c2ec21dc admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:39.259195 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ffa07d-71f0-4b78-a88d-2d17c2ec21dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:39.259482 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ffa07d-71f0-4b78-a88d-2d17c2ec21dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:39.260651 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ffa07d-71f0-4b78-a88d-2d17c2ec21dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:39.263688 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ffa07d-71f0-4b78-a88d-2d17c2ec21dc admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:39.280481 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1370/2732] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:28:39.281951 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eab50e-5eb2-4f0f-ab45-850423a3f9f4 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:39.311094 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eab50e-5eb2-4f0f-ab45-850423a3f9f4 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:39.317255 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a80e7f0-7659-4b75-b93d-830ccaf4de12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:39.318182 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80e7f0-7659-4b75-b93d-830ccaf4de12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:39.318469 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80e7f0-7659-4b75-b93d-830ccaf4de12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:39.318750 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80e7f0-7659-4b75-b93d-830ccaf4de12 admin admin] PATH_INFO: `/v3/projects/0d66b500a89e4241a53a766d931ff4c9/users/fea092c596874c15ae5574d5b009120a/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:39.337609 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a80e7f0-7659-4b75-b93d-830ccaf4de12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d66b500a89e4241a53a766d931ff4c9, user_id=fea092c596874c15ae5574d5b009120a, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:39.342207 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a80e7f0-7659-4b75-b93d-830ccaf4de12 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:39.375573 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1363/2733] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:28:39 2022] PUT /identity/v3/projects/0d66b500a89e4241a53a766d931ff4c9/users/fea092c596874c15ae5574d5b009120a/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:39.380128 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ffa07d-71f0-4b78-a88d-2d17c2ec21dc admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:39.397131 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ffa07d-71f0-4b78-a88d-2d17c2ec21dc admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:39.487731 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a80e7f0-7659-4b75-b93d-830ccaf4de12 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:39.489721 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7a80e7f0-7659-4b75-b93d-830ccaf4de12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:39.506890 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2811010a-fc1b-4aef-a475-4551aada60a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:39.508911 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811010a-fc1b-4aef-a475-4551aada60a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:39.509263 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811010a-fc1b-4aef-a475-4551aada60a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:39.509542 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811010a-fc1b-4aef-a475-4551aada60a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:39.510339 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2811010a-fc1b-4aef-a475-4551aada60a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:39.513569 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2811010a-fc1b-4aef-a475-4551aada60a6 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:39.531508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1371/2734] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:28:39.535642 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2811010a-fc1b-4aef-a475-4551aada60a6 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:39.552560 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2811010a-fc1b-4aef-a475-4551aada60a6 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:39.567965 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5dcdd7-cf73-43e7-b0ac-a20c7589a704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:39.569539 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5dcdd7-cf73-43e7-b0ac-a20c7589a704 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:39.569827 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5dcdd7-cf73-43e7-b0ac-a20c7589a704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:39.570374 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5dcdd7-cf73-43e7-b0ac-a20c7589a704 admin admin] PATH_INFO: `/v3/projects/0d66b500a89e4241a53a766d931ff4c9/users/fea092c596874c15ae5574d5b009120a/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:39.573354 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5dcdd7-cf73-43e7-b0ac-a20c7589a704 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d66b500a89e4241a53a766d931ff4c9, user_id=fea092c596874c15ae5574d5b009120a, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:39.576802 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5dcdd7-cf73-43e7-b0ac-a20c7589a704 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:39.578607 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a80e7f0-7659-4b75-b93d-830ccaf4de12 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:39.586528 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc1b80c-82d8-49a8-a79b-ccd1282c3a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:39.588184 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc1b80c-82d8-49a8-a79b-ccd1282c3a41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:39.588736 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc1b80c-82d8-49a8-a79b-ccd1282c3a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:39.589251 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc1b80c-82d8-49a8-a79b-ccd1282c3a41 admin admin] PATH_INFO: `/v3/projects/f7316f34b0e24550a3faf7718061ad80` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:39.590870 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc1b80c-82d8-49a8-a79b-ccd1282c3a41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7316f34b0e24550a3faf7718061ad80)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:39.591060 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1372/2735] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:28:39 2022] PUT /identity/v3/projects/0d66b500a89e4241a53a766d931ff4c9/users/fea092c596874c15ae5574d5b009120a/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:28:39.600664 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc1b80c-82d8-49a8-a79b-ccd1282c3a41 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:39.604626 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398345ea-5c0e-4f0b-a091-dd4391fc5e2d None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:39.605037 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398345ea-5c0e-4f0b-a091-dd4391fc5e2d None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:39.605339 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398345ea-5c0e-4f0b-a091-dd4391fc5e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:39.645685 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-398345ea-5c0e-4f0b-a091-dd4391fc5e2d None None] MFA Rules not processed for user `fea092c596874c15ae5574d5b009120a`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:39.647906 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-398345ea-5c0e-4f0b-a091-dd4391fc5e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:39.675466 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-bdc1b80c-82d8-49a8-a79b-ccd1282c3a41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7316f34b0e24550a3faf7718061ad80', 'request_id': 'req-bdc1b80c-82d8-49a8-a79b-ccd1282c3a41'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:39.685578 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-bdc1b80c-82d8-49a8-a79b-ccd1282c3a41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7316f34b0e24550a3faf7718061ad80', 'request_id': 'req-bdc1b80c-82d8-49a8-a79b-ccd1282c3a41'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:39.697291 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-bdc1b80c-82d8-49a8-a79b-ccd1282c3a41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7316f34b0e24550a3faf7718061ad80', 'request_id': 'req-bdc1b80c-82d8-49a8-a79b-ccd1282c3a41'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:39.716986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-bdc1b80c-82d8-49a8-a79b-ccd1282c3a41 admin admin] The token cache is being invalidate because project f7316f34b0e24550a3faf7718061ad80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:28:39.717424 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-bdc1b80c-82d8-49a8-a79b-ccd1282c3a41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:39.718870 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-bdc1b80c-82d8-49a8-a79b-ccd1282c3a41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:39.721880 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1364/2736] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:28:39 2022] DELETE /identity/v3/projects/f7316f34b0e24550a3faf7718061ad80 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:39.726014 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc1b80c-82d8-49a8-a79b-ccd1282c3a41 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:39.727865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-bdc1b80c-82d8-49a8-a79b-ccd1282c3a41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:39.775377 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1373/2737] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:39 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:28:39.791563 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398345ea-5c0e-4f0b-a091-dd4391fc5e2d None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:39.792787 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-398345ea-5c0e-4f0b-a091-dd4391fc5e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:39.840733 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc1b80c-82d8-49a8-a79b-ccd1282c3a41 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:39.842730 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:39.848648 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae6b91-a81b-4635-b6f0-232ebe834185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:39.850870 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae6b91-a81b-4635-b6f0-232ebe834185 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:39.853021 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae6b91-a81b-4635-b6f0-232ebe834185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:39.855058 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae6b91-a81b-4635-b6f0-232ebe834185 admin admin] PATH_INFO: `/v3/users/a113fdc7212f496896934da861a8fbe6` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:39.857235 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae6b91-a81b-4635-b6f0-232ebe834185 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a113fdc7212f496896934da861a8fbe6)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:39.859608 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae6b91-a81b-4635-b6f0-232ebe834185 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:39.862725 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:39.893134 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398345ea-5c0e-4f0b-a091-dd4391fc5e2d None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:39.938124 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8aae6b91-a81b-4635-b6f0-232ebe834185 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a113fdc7212f496896934da861a8fbe6'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:39.939343 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8aae6b91-a81b-4635-b6f0-232ebe834185 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a113fdc7212f496896934da861a8fbe6'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:39.955110 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263b021-5c49-4d17-8414-6a99335e10f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:39.955862 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263b021-5c49-4d17-8414-6a99335e10f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:39.956293 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263b021-5c49-4d17-8414-6a99335e10f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:39.956630 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263b021-5c49-4d17-8414-6a99335e10f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:39.959064 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-1263b021-5c49-4d17-8414-6a99335e10f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:39.960794 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8aae6b91-a81b-4635-b6f0-232ebe834185 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a113fdc7212f496896934da861a8fbe6'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:39.977978 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8aae6b91-a81b-4635-b6f0-232ebe834185 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a113fdc7212f496896934da861a8fbe6'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:39.989285 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8aae6b91-a81b-4635-b6f0-232ebe834185 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a113fdc7212f496896934da861a8fbe6'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:40.001059 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1365/2738] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:28:39 2022] DELETE /identity/v3/users/a113fdc7212f496896934da861a8fbe6 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:40.099383 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1263b021-5c49-4d17-8414-6a99335e10f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:40.102209 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1263b021-5c49-4d17-8414-6a99335e10f4 service cinder] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:40.132248 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1374/2739] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:39 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:28:40.198151 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae6b91-a81b-4635-b6f0-232ebe834185 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:40.199664 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8aae6b91-a81b-4635-b6f0-232ebe834185 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:40.328176 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae6b91-a81b-4635-b6f0-232ebe834185 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:40.332388 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf80de-98b0-486d-b581-fdf24d47a40c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:40.336639 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf80de-98b0-486d-b581-fdf24d47a40c service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:40.336927 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf80de-98b0-486d-b581-fdf24d47a40c service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:40.342554 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf80de-98b0-486d-b581-fdf24d47a40c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:40.344750 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-22bf80de-98b0-486d-b581-fdf24d47a40c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:40.429052 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bf80de-98b0-486d-b581-fdf24d47a40c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:40.431282 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bf80de-98b0-486d-b581-fdf24d47a40c service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:40.442342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1366/2740] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:28:40.535747 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263b021-5c49-4d17-8414-6a99335e10f4 service cinder] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:40.538595 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-1263b021-5c49-4d17-8414-6a99335e10f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:40.578418 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263b021-5c49-4d17-8414-6a99335e10f4 service cinder] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:40.698676 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041536e2-d713-4fd2-b1d6-cad50f563753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:40.700245 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041536e2-d713-4fd2-b1d6-cad50f563753 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:40.700510 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041536e2-d713-4fd2-b1d6-cad50f563753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:40.701080 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041536e2-d713-4fd2-b1d6-cad50f563753 admin admin] PATH_INFO: `/v3/projects/9c4545b376b540b4bd9c82dceb1c1d56` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:40.702407 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041536e2-d713-4fd2-b1d6-cad50f563753 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c4545b376b540b4bd9c82dceb1c1d56)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:40.704563 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041536e2-d713-4fd2-b1d6-cad50f563753 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:40.759003 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:40.778912 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-041536e2-d713-4fd2-b1d6-cad50f563753 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c4545b376b540b4bd9c82dceb1c1d56', 'request_id': 'req-041536e2-d713-4fd2-b1d6-cad50f563753'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:40.789083 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-041536e2-d713-4fd2-b1d6-cad50f563753 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c4545b376b540b4bd9c82dceb1c1d56', 'request_id': 'req-041536e2-d713-4fd2-b1d6-cad50f563753'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:40.805189 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-041536e2-d713-4fd2-b1d6-cad50f563753 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c4545b376b540b4bd9c82dceb1c1d56', 'request_id': 'req-041536e2-d713-4fd2-b1d6-cad50f563753'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:40.820759 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-041536e2-d713-4fd2-b1d6-cad50f563753 admin admin] The token cache is being invalidate because project 9c4545b376b540b4bd9c82dceb1c1d56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:28:40.821040 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-041536e2-d713-4fd2-b1d6-cad50f563753 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:40.822509 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-041536e2-d713-4fd2-b1d6-cad50f563753 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:40.825113 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1375/2741] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:28:40 2022] DELETE /identity/v3/projects/9c4545b376b540b4bd9c82dceb1c1d56 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:40.829735 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf80de-98b0-486d-b581-fdf24d47a40c service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:40.841360 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf80de-98b0-486d-b581-fdf24d47a40c service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:40.892906 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:40.958032 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b1a1b-2d17-4afc-be90-090bf13d2623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:40.958032 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b1a1b-2d17-4afc-be90-090bf13d2623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:40.958032 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b1a1b-2d17-4afc-be90-090bf13d2623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:40.958032 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b1a1b-2d17-4afc-be90-090bf13d2623 admin admin] PATH_INFO: `/v3/users/de6c1b5c8f7c427c9d726b26222386cb` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:40.959226 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5b1a1b-2d17-4afc-be90-090bf13d2623 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de6c1b5c8f7c427c9d726b26222386cb)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:40.959226 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5b1a1b-2d17-4afc-be90-090bf13d2623 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:41.010316 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2f5b1a1b-2d17-4afc-be90-090bf13d2623 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de6c1b5c8f7c427c9d726b26222386cb'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:41.011134 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2f5b1a1b-2d17-4afc-be90-090bf13d2623 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de6c1b5c8f7c427c9d726b26222386cb'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:41.026213 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2f5b1a1b-2d17-4afc-be90-090bf13d2623 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de6c1b5c8f7c427c9d726b26222386cb'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:41.033766 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2f5b1a1b-2d17-4afc-be90-090bf13d2623 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de6c1b5c8f7c427c9d726b26222386cb'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:41.041075 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-2f5b1a1b-2d17-4afc-be90-090bf13d2623 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de6c1b5c8f7c427c9d726b26222386cb'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:41.048693 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1367/2742] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:28:40 2022] DELETE /identity/v3/users/de6c1b5c8f7c427c9d726b26222386cb => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:41.340911 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041536e2-d713-4fd2-b1d6-cad50f563753 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:41.342565 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-041536e2-d713-4fd2-b1d6-cad50f563753 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:41.361652 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041536e2-d713-4fd2-b1d6-cad50f563753 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:41.468905 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c39f12f-7440-4ed3-bc07-260c2297d1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:41.469789 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c39f12f-7440-4ed3-bc07-260c2297d1fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:41.470074 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c39f12f-7440-4ed3-bc07-260c2297d1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:41.470354 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c39f12f-7440-4ed3-bc07-260c2297d1fb admin admin] PATH_INFO: `/v3/projects/48fd1a09060349a5a971bffa83a283b3` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:41.471603 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c39f12f-7440-4ed3-bc07-260c2297d1fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=48fd1a09060349a5a971bffa83a283b3)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:41.473703 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c39f12f-7440-4ed3-bc07-260c2297d1fb admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:41.527009 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-4c39f12f-7440-4ed3-bc07-260c2297d1fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48fd1a09060349a5a971bffa83a283b3', 'request_id': 'req-4c39f12f-7440-4ed3-bc07-260c2297d1fb'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:41.535814 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-4c39f12f-7440-4ed3-bc07-260c2297d1fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48fd1a09060349a5a971bffa83a283b3', 'request_id': 'req-4c39f12f-7440-4ed3-bc07-260c2297d1fb'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:41.545681 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-4c39f12f-7440-4ed3-bc07-260c2297d1fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48fd1a09060349a5a971bffa83a283b3', 'request_id': 'req-4c39f12f-7440-4ed3-bc07-260c2297d1fb'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:41.557089 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-4c39f12f-7440-4ed3-bc07-260c2297d1fb admin admin] The token cache is being invalidate because project 48fd1a09060349a5a971bffa83a283b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:28:41.557386 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-4c39f12f-7440-4ed3-bc07-260c2297d1fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:41.558051 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-4c39f12f-7440-4ed3-bc07-260c2297d1fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:41.559680 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1376/2743] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:28:41 2022] DELETE /identity/v3/projects/48fd1a09060349a5a971bffa83a283b3 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:41.565715 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e460af04-bfb9-4942-af9e-58c9c96676cd None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:41.566270 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e460af04-bfb9-4942-af9e-58c9c96676cd None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:41.566270 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e460af04-bfb9-4942-af9e-58c9c96676cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:41.584941 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-e460af04-bfb9-4942-af9e-58c9c96676cd None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:41.586170 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e460af04-bfb9-4942-af9e-58c9c96676cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:41.651989 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1368/2744] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:28:41.661004 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece58d92-1b32-4668-bf79-da55c936d96f None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:41.661282 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece58d92-1b32-4668-bf79-da55c936d96f None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:41.661568 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece58d92-1b32-4668-bf79-da55c936d96f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:41.682591 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-ece58d92-1b32-4668-bf79-da55c936d96f None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:41.683908 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-ece58d92-1b32-4668-bf79-da55c936d96f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:41.692491 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1377/2745] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:41.696887 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e460af04-bfb9-4942-af9e-58c9c96676cd None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:41.708954 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e460af04-bfb9-4942-af9e-58c9c96676cd None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:41.719369 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595067a-a357-4266-a48f-5c0b262e42bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:41.720367 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595067a-a357-4266-a48f-5c0b262e42bc admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:41.720747 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595067a-a357-4266-a48f-5c0b262e42bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:41.721027 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595067a-a357-4266-a48f-5c0b262e42bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:41.721928 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a595067a-a357-4266-a48f-5c0b262e42bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:28:41.722239 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a595067a-a357-4266-a48f-5c0b262e42bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:41.723947 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a595067a-a357-4266-a48f-5c0b262e42bc admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:41.737487 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1369/2746] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:28:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:28:41.744228 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece58d92-1b32-4668-bf79-da55c936d96f None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:41.754467 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece58d92-1b32-4668-bf79-da55c936d96f None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:41.759471 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f859a7a-158d-45aa-bfb2-6edbc2cd24f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:41.760199 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f859a7a-158d-45aa-bfb2-6edbc2cd24f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:41.766007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f859a7a-158d-45aa-bfb2-6edbc2cd24f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:41.766007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f859a7a-158d-45aa-bfb2-6edbc2cd24f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:41.766007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f859a7a-158d-45aa-bfb2-6edbc2cd24f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:41.766741 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f859a7a-158d-45aa-bfb2-6edbc2cd24f6 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:41.794294 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1378/2747] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:28:41 2022] POST /identity/v3/projects => generated 399 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:28:41.798807 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595067a-a357-4266-a48f-5c0b262e42bc admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:41.810183 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595067a-a357-4266-a48f-5c0b262e42bc admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:41.857870 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:41.877681 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ae0e9-23d5-40ab-9fac-c220de04acaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:41.878357 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ae0e9-23d5-40ab-9fac-c220de04acaa admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:41.878636 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ae0e9-23d5-40ab-9fac-c220de04acaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:41.878905 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ae0e9-23d5-40ab-9fac-c220de04acaa admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:41.879640 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ae0e9-23d5-40ab-9fac-c220de04acaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:41.881590 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ae0e9-23d5-40ab-9fac-c220de04acaa admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:41.907302 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1370/2748] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:28:41 2022] POST /identity/v3/users => generated 300 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:28:41.911470 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f859a7a-158d-45aa-bfb2-6edbc2cd24f6 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:41.922773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f859a7a-158d-45aa-bfb2-6edbc2cd24f6 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:41.927508 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c92fe9c-73bf-4678-832b-7ae006504592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:41.928199 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c92fe9c-73bf-4678-832b-7ae006504592 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:41.928476 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c92fe9c-73bf-4678-832b-7ae006504592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:41.928773 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c92fe9c-73bf-4678-832b-7ae006504592 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:41.929537 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c92fe9c-73bf-4678-832b-7ae006504592 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:41.931286 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c92fe9c-73bf-4678-832b-7ae006504592 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:41.947981 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1379/2749] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:28:41.952346 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ae0e9-23d5-40ab-9fac-c220de04acaa admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:41.964071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ae0e9-23d5-40ab-9fac-c220de04acaa admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:41.971310 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d32cc-0c32-4f23-b500-21fb6be15a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:41.972048 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d32cc-0c32-4f23-b500-21fb6be15a89 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:41.972321 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d32cc-0c32-4f23-b500-21fb6be15a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:41.972594 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d32cc-0c32-4f23-b500-21fb6be15a89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:41.973363 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d32cc-0c32-4f23-b500-21fb6be15a89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:41.974983 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d32cc-0c32-4f23-b500-21fb6be15a89 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:41.987229 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1371/2750] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:28:41.991334 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c92fe9c-73bf-4678-832b-7ae006504592 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:42.016483 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c92fe9c-73bf-4678-832b-7ae006504592 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:42.025422 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2905baa0-cbbf-45f1-9fba-b851f443740b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:42.026487 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905baa0-cbbf-45f1-9fba-b851f443740b admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:42.026837 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905baa0-cbbf-45f1-9fba-b851f443740b admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:42.027980 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905baa0-cbbf-45f1-9fba-b851f443740b admin admin] PATH_INFO: `/v3/projects/9f97cb0b78d241828644d3722ec1377e/users/c819429c1e2f4fa18abda4402277a125/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:42.032568 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:42.050940 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2905baa0-cbbf-45f1-9fba-b851f443740b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f97cb0b78d241828644d3722ec1377e, user_id=c819429c1e2f4fa18abda4402277a125, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:42.053812 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2905baa0-cbbf-45f1-9fba-b851f443740b admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:42.076595 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1380/2751] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:28:41 2022] PUT /identity/v3/projects/9f97cb0b78d241828644d3722ec1377e/users/c819429c1e2f4fa18abda4402277a125/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:42.088915 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb05a6-ea2f-43aa-becd-ebcabebc1da9 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:42.089251 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb05a6-ea2f-43aa-becd-ebcabebc1da9 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:42.089549 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb05a6-ea2f-43aa-becd-ebcabebc1da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:42.120912 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-9afb05a6-ea2f-43aa-becd-ebcabebc1da9 None None] MFA Rules not processed for user `c819429c1e2f4fa18abda4402277a125`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:42.122335 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9afb05a6-ea2f-43aa-becd-ebcabebc1da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:42.211830 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1372/2752] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:42 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:28:42.216832 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2905baa0-cbbf-45f1-9fba-b851f443740b admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:42.300409 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2905baa0-cbbf-45f1-9fba-b851f443740b admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:42.309072 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c2155-eaf8-4e47-8430-de7b521f1bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:42.309072 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c2155-eaf8-4e47-8430-de7b521f1bb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:42.309072 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c2155-eaf8-4e47-8430-de7b521f1bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:42.309072 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c2155-eaf8-4e47-8430-de7b521f1bb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:42.309610 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8c2155-eaf8-4e47-8430-de7b521f1bb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:42.311059 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8c2155-eaf8-4e47-8430-de7b521f1bb6 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:42.340880 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1381/2753] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:28:42 2022] POST /identity/v3/projects => generated 399 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:28:42.345297 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb05a6-ea2f-43aa-becd-ebcabebc1da9 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:42.369589 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb05a6-ea2f-43aa-becd-ebcabebc1da9 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:42.438485 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a52d3-0426-4707-a110-92f1e6b3987a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:42.439180 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a52d3-0426-4707-a110-92f1e6b3987a admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:42.439465 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a52d3-0426-4707-a110-92f1e6b3987a admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:42.439773 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a52d3-0426-4707-a110-92f1e6b3987a admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:42.440462 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a52d3-0426-4707-a110-92f1e6b3987a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:42.442409 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a52d3-0426-4707-a110-92f1e6b3987a admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:42.465023 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1373/2754] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:28:42 2022] POST /identity/v3/users => generated 306 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:28:42.466130 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c2155-eaf8-4e47-8430-de7b521f1bb6 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:42.479601 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c2155-eaf8-4e47-8430-de7b521f1bb6 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:42.484512 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010f4f1-d610-4a38-84ea-6ce6cf65d5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:42.485181 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010f4f1-d610-4a38-84ea-6ce6cf65d5e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:42.485467 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010f4f1-d610-4a38-84ea-6ce6cf65d5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:42.485752 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010f4f1-d610-4a38-84ea-6ce6cf65d5e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:42.486484 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8010f4f1-d610-4a38-84ea-6ce6cf65d5e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:42.488135 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8010f4f1-d610-4a38-84ea-6ce6cf65d5e8 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:42.499891 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1382/2755] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:28:42.503915 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a52d3-0426-4707-a110-92f1e6b3987a admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:42.514493 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a52d3-0426-4707-a110-92f1e6b3987a admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:42.524502 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2456cd-29a7-492b-8f56-00fdfe6640f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:42.525401 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2456cd-29a7-492b-8f56-00fdfe6640f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:42.525682 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2456cd-29a7-492b-8f56-00fdfe6640f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:42.526914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2456cd-29a7-492b-8f56-00fdfe6640f9 admin admin] PATH_INFO: `/v3/projects/00f76bf8bca8441abcb522d398e28fb3/users/06b52da53c224d97a8eb1f17064f46a6/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:42.542157 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2456cd-29a7-492b-8f56-00fdfe6640f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00f76bf8bca8441abcb522d398e28fb3, user_id=06b52da53c224d97a8eb1f17064f46a6, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:42.545243 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2456cd-29a7-492b-8f56-00fdfe6640f9 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:42.561193 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1374/2756] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:28:42 2022] PUT /identity/v3/projects/00f76bf8bca8441abcb522d398e28fb3/users/06b52da53c224d97a8eb1f17064f46a6/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:42.566067 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010f4f1-d610-4a38-84ea-6ce6cf65d5e8 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:42.578305 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010f4f1-d610-4a38-84ea-6ce6cf65d5e8 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:42.644109 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6feba53-07b3-4524-83b5-ad143d3851e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:42.645547 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6feba53-07b3-4524-83b5-ad143d3851e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:42.645830 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6feba53-07b3-4524-83b5-ad143d3851e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:42.646087 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6feba53-07b3-4524-83b5-ad143d3851e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:42.646806 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6feba53-07b3-4524-83b5-ad143d3851e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:42.648419 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6feba53-07b3-4524-83b5-ad143d3851e1 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:42.658955 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1383/2757] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:28:42.663166 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2456cd-29a7-492b-8f56-00fdfe6640f9 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:42.674913 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2456cd-29a7-492b-8f56-00fdfe6640f9 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:42.680886 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5002045b-b5bf-4783-a648-bd16d36506d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:42.681530 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5002045b-b5bf-4783-a648-bd16d36506d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:42.681875 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5002045b-b5bf-4783-a648-bd16d36506d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:42.682212 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5002045b-b5bf-4783-a648-bd16d36506d2 admin admin] PATH_INFO: `/v3/projects/00f76bf8bca8441abcb522d398e28fb3/users/06b52da53c224d97a8eb1f17064f46a6/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:42.685512 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5002045b-b5bf-4783-a648-bd16d36506d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00f76bf8bca8441abcb522d398e28fb3, user_id=06b52da53c224d97a8eb1f17064f46a6, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:42.688712 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5002045b-b5bf-4783-a648-bd16d36506d2 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:42.702185 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1375/2758] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:28:42 2022] PUT /identity/v3/projects/00f76bf8bca8441abcb522d398e28fb3/users/06b52da53c224d97a8eb1f17064f46a6/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:42.712097 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ffa26-bd4b-4e13-ab89-964c8abaac49 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:42.712409 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ffa26-bd4b-4e13-ab89-964c8abaac49 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:42.712799 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ffa26-bd4b-4e13-ab89-964c8abaac49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:42.753195 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-da9ffa26-bd4b-4e13-ab89-964c8abaac49 None None] MFA Rules not processed for user `06b52da53c224d97a8eb1f17064f46a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:42.754616 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-da9ffa26-bd4b-4e13-ab89-964c8abaac49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:42.815255 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:42.854576 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1384/2759] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:42 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:42.869313 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5002045b-b5bf-4783-a648-bd16d36506d2 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:42.873234 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5002045b-b5bf-4783-a648-bd16d36506d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:42.879997 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:42.984024 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5002045b-b5bf-4783-a648-bd16d36506d2 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:42.988680 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca8a3c4-16ca-413f-a559-31f2c8a999d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:42.989126 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8a3c4-16ca-413f-a559-31f2c8a999d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:42.989473 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8a3c4-16ca-413f-a559-31f2c8a999d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:42.989858 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8a3c4-16ca-413f-a559-31f2c8a999d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:43.001123 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca8a3c4-16ca-413f-a559-31f2c8a999d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:43.003449 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca8a3c4-16ca-413f-a559-31f2c8a999d1 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:43.013790 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1376/2760] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:42 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:28:43.096802 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9ffa26-bd4b-4e13-ab89-964c8abaac49 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:43.098407 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-da9ffa26-bd4b-4e13-ab89-964c8abaac49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:43.111900 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9ffa26-bd4b-4e13-ab89-964c8abaac49 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:43.116057 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ebf0d-beeb-459c-9404-7a9376d90c33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:43.116664 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ebf0d-beeb-459c-9404-7a9376d90c33 service cinder] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:43.116912 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ebf0d-beeb-459c-9404-7a9376d90c33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:43.117188 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ebf0d-beeb-459c-9404-7a9376d90c33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:43.178046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568ebf0d-beeb-459c-9404-7a9376d90c33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:43.179637 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568ebf0d-beeb-459c-9404-7a9376d90c33 service cinder] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:43.189227 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1385/2761] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:43 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:28:44.174835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca8a3c4-16ca-413f-a559-31f2c8a999d1 service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:44.176425 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4ca8a3c4-16ca-413f-a559-31f2c8a999d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:44.246489 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca8a3c4-16ca-413f-a559-31f2c8a999d1 service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:44.251088 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accdf3c6-74a6-4d37-ae39-b466d358568f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:44.252218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accdf3c6-74a6-4d37-ae39-b466d358568f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:44.252504 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accdf3c6-74a6-4d37-ae39-b466d358568f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:44.252779 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accdf3c6-74a6-4d37-ae39-b466d358568f admin admin] PATH_INFO: `/v3/users/b2dbd3c8cbf242bcb5b3a8d1931fcb15` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:44.254338 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accdf3c6-74a6-4d37-ae39-b466d358568f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2dbd3c8cbf242bcb5b3a8d1931fcb15)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:44.256591 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accdf3c6-74a6-4d37-ae39-b466d358568f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:44.304292 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-accdf3c6-74a6-4d37-ae39-b466d358568f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2dbd3c8cbf242bcb5b3a8d1931fcb15'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:44.304950 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-accdf3c6-74a6-4d37-ae39-b466d358568f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2dbd3c8cbf242bcb5b3a8d1931fcb15'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:44.316602 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-accdf3c6-74a6-4d37-ae39-b466d358568f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2dbd3c8cbf242bcb5b3a8d1931fcb15'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:44.323414 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-accdf3c6-74a6-4d37-ae39-b466d358568f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2dbd3c8cbf242bcb5b3a8d1931fcb15'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:44.331447 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-accdf3c6-74a6-4d37-ae39-b466d358568f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2dbd3c8cbf242bcb5b3a8d1931fcb15'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:44.337431 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1377/2762] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:28:44 2022] DELETE /identity/v3/users/b2dbd3c8cbf242bcb5b3a8d1931fcb15 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:44.342720 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3ee84-027a-4cef-9a8b-c9e65a8f4c16 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:44.343251 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3ee84-027a-4cef-9a8b-c9e65a8f4c16 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:44.343670 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3ee84-027a-4cef-9a8b-c9e65a8f4c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:44.361018 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-9ce3ee84-027a-4cef-9a8b-c9e65a8f4c16 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:44.362314 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-9ce3ee84-027a-4cef-9a8b-c9e65a8f4c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:44.427456 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1386/2763] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:44.437157 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accdf3c6-74a6-4d37-ae39-b466d358568f admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:44.438632 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-accdf3c6-74a6-4d37-ae39-b466d358568f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:44.469982 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:44.503930 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accdf3c6-74a6-4d37-ae39-b466d358568f admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:44.507688 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254dbcac-ebc4-4a61-b2f4-619cc4fe8bb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:44.508344 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254dbcac-ebc4-4a61-b2f4-619cc4fe8bb5 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:44.508654 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254dbcac-ebc4-4a61-b2f4-619cc4fe8bb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:44.508911 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254dbcac-ebc4-4a61-b2f4-619cc4fe8bb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:44.520061 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254dbcac-ebc4-4a61-b2f4-619cc4fe8bb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:44.521533 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254dbcac-ebc4-4a61-b2f4-619cc4fe8bb5 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:44.535731 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1378/2764] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:28:44.859784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3ee84-027a-4cef-9a8b-c9e65a8f4c16 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:44.861718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-9ce3ee84-027a-4cef-9a8b-c9e65a8f4c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:44.878062 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3ee84-027a-4cef-9a8b-c9e65a8f4c16 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:44.882970 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a95a431-a2c9-4603-ad49-afa6d2a47486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:44.883898 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95a431-a2c9-4603-ad49-afa6d2a47486 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:44.884183 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95a431-a2c9-4603-ad49-afa6d2a47486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:44.884536 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95a431-a2c9-4603-ad49-afa6d2a47486 admin admin] PATH_INFO: `/v3/projects/81a13212db634e53bfe355900e90bd9c` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:44.885721 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a95a431-a2c9-4603-ad49-afa6d2a47486 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81a13212db634e53bfe355900e90bd9c)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:44.888561 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a95a431-a2c9-4603-ad49-afa6d2a47486 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:44.941073 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-0a95a431-a2c9-4603-ad49-afa6d2a47486 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81a13212db634e53bfe355900e90bd9c', 'request_id': 'req-0a95a431-a2c9-4603-ad49-afa6d2a47486'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:44.947251 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-0a95a431-a2c9-4603-ad49-afa6d2a47486 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81a13212db634e53bfe355900e90bd9c', 'request_id': 'req-0a95a431-a2c9-4603-ad49-afa6d2a47486'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:44.955647 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-0a95a431-a2c9-4603-ad49-afa6d2a47486 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81a13212db634e53bfe355900e90bd9c', 'request_id': 'req-0a95a431-a2c9-4603-ad49-afa6d2a47486'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:44.966851 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-0a95a431-a2c9-4603-ad49-afa6d2a47486 admin admin] The token cache is being invalidate because project 81a13212db634e53bfe355900e90bd9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:28:44.967201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-0a95a431-a2c9-4603-ad49-afa6d2a47486 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:44.967916 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-0a95a431-a2c9-4603-ad49-afa6d2a47486 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:44.969637 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1387/2765] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:28:44 2022] DELETE /identity/v3/projects/81a13212db634e53bfe355900e90bd9c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:44.975877 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254dbcac-ebc4-4a61-b2f4-619cc4fe8bb5 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:44.991476 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254dbcac-ebc4-4a61-b2f4-619cc4fe8bb5 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:45.050739 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba960dd-ec2d-4b25-96e7-504f03dcded8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:45.051633 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba960dd-ec2d-4b25-96e7-504f03dcded8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:45.051915 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba960dd-ec2d-4b25-96e7-504f03dcded8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:45.052190 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba960dd-ec2d-4b25-96e7-504f03dcded8 admin admin] PATH_INFO: `/v3/users/fea092c596874c15ae5574d5b009120a` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:45.053381 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba960dd-ec2d-4b25-96e7-504f03dcded8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fea092c596874c15ae5574d5b009120a)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:45.055298 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba960dd-ec2d-4b25-96e7-504f03dcded8 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:45.102056 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8ba960dd-ec2d-4b25-96e7-504f03dcded8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fea092c596874c15ae5574d5b009120a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:45.102728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8ba960dd-ec2d-4b25-96e7-504f03dcded8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fea092c596874c15ae5574d5b009120a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:45.115675 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8ba960dd-ec2d-4b25-96e7-504f03dcded8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fea092c596874c15ae5574d5b009120a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:45.123831 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8ba960dd-ec2d-4b25-96e7-504f03dcded8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fea092c596874c15ae5574d5b009120a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:45.133635 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-8ba960dd-ec2d-4b25-96e7-504f03dcded8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fea092c596874c15ae5574d5b009120a'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:45.140627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1379/2766] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:28:44 2022] DELETE /identity/v3/users/fea092c596874c15ae5574d5b009120a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:45.422193 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a95a431-a2c9-4603-ad49-afa6d2a47486 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:45.423750 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-0a95a431-a2c9-4603-ad49-afa6d2a47486 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:45.431937 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:45.437477 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a95a431-a2c9-4603-ad49-afa6d2a47486 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:45.499102 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39d7d1e-5a43-418b-b8d0-a62312db0372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:45.499889 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d7d1e-5a43-418b-b8d0-a62312db0372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:45.500172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d7d1e-5a43-418b-b8d0-a62312db0372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:45.500448 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d7d1e-5a43-418b-b8d0-a62312db0372 admin admin] PATH_INFO: `/v3/projects/0d66b500a89e4241a53a766d931ff4c9` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:45.501609 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39d7d1e-5a43-418b-b8d0-a62312db0372 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d66b500a89e4241a53a766d931ff4c9)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:45.503574 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39d7d1e-5a43-418b-b8d0-a62312db0372 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:45.555323 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e39d7d1e-5a43-418b-b8d0-a62312db0372 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d66b500a89e4241a53a766d931ff4c9', 'request_id': 'req-e39d7d1e-5a43-418b-b8d0-a62312db0372'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:45.562198 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e39d7d1e-5a43-418b-b8d0-a62312db0372 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d66b500a89e4241a53a766d931ff4c9', 'request_id': 'req-e39d7d1e-5a43-418b-b8d0-a62312db0372'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:45.569930 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e39d7d1e-5a43-418b-b8d0-a62312db0372 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d66b500a89e4241a53a766d931ff4c9', 'request_id': 'req-e39d7d1e-5a43-418b-b8d0-a62312db0372'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:45.580940 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e39d7d1e-5a43-418b-b8d0-a62312db0372 admin admin] The token cache is being invalidate because project 0d66b500a89e4241a53a766d931ff4c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:28:45.581230 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e39d7d1e-5a43-418b-b8d0-a62312db0372 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:45.581889 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e39d7d1e-5a43-418b-b8d0-a62312db0372 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:45.583439 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1388/2767] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:28:45 2022] DELETE /identity/v3/projects/0d66b500a89e4241a53a766d931ff4c9 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:45.590419 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d42fd1a-e3ba-4575-9cdd-524f3fea3a02 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:45.590902 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d42fd1a-e3ba-4575-9cdd-524f3fea3a02 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:45.591025 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d42fd1a-e3ba-4575-9cdd-524f3fea3a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:45.610064 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-1d42fd1a-e3ba-4575-9cdd-524f3fea3a02 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:45.611290 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1d42fd1a-e3ba-4575-9cdd-524f3fea3a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:45.659489 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:45.679003 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1380/2768] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:45.687265 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1b3fa-5a7b-4ae4-beb6-7eb026bc3ae7 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:45.687603 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1b3fa-5a7b-4ae4-beb6-7eb026bc3ae7 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:45.687893 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1b3fa-5a7b-4ae4-beb6-7eb026bc3ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:45.706943 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-71a1b3fa-5a7b-4ae4-beb6-7eb026bc3ae7 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:45.708173 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-71a1b3fa-5a7b-4ae4-beb6-7eb026bc3ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:45.716682 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1389/2769] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:28:45.720990 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d42fd1a-e3ba-4575-9cdd-524f3fea3a02 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:45.730629 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d42fd1a-e3ba-4575-9cdd-524f3fea3a02 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:45.735476 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995d5a3d-0311-45ad-9fd8-a8ad402ec233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:45.736171 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d5a3d-0311-45ad-9fd8-a8ad402ec233 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:45.736450 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d5a3d-0311-45ad-9fd8-a8ad402ec233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:45.736726 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d5a3d-0311-45ad-9fd8-a8ad402ec233 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:45.737452 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995d5a3d-0311-45ad-9fd8-a8ad402ec233 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:28:45.737751 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995d5a3d-0311-45ad-9fd8-a8ad402ec233 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:45.739340 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995d5a3d-0311-45ad-9fd8-a8ad402ec233 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:45.751361 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1381/2770] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:28:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:28:45.755679 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a1b3fa-5a7b-4ae4-beb6-7eb026bc3ae7 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:45.765630 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a1b3fa-5a7b-4ae4-beb6-7eb026bc3ae7 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:45.770312 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de054b-15df-4a96-b150-ba32ff35c4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:45.770960 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de054b-15df-4a96-b150-ba32ff35c4a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:45.771242 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de054b-15df-4a96-b150-ba32ff35c4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:45.771517 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de054b-15df-4a96-b150-ba32ff35c4a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:45.772244 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26de054b-15df-4a96-b150-ba32ff35c4a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:45.774172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26de054b-15df-4a96-b150-ba32ff35c4a2 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:45.804666 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1390/2771] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:28:45 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:28:45.809001 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995d5a3d-0311-45ad-9fd8-a8ad402ec233 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:45.822775 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995d5a3d-0311-45ad-9fd8-a8ad402ec233 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:45.881057 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb681a1a-c596-4ea6-bfcc-9112c72f2cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:45.881732 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb681a1a-c596-4ea6-bfcc-9112c72f2cd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:45.882012 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb681a1a-c596-4ea6-bfcc-9112c72f2cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:45.882286 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb681a1a-c596-4ea6-bfcc-9112c72f2cd4 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:45.882979 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb681a1a-c596-4ea6-bfcc-9112c72f2cd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:45.884966 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb681a1a-c596-4ea6-bfcc-9112c72f2cd4 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:45.908468 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1382/2772] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:28:45 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:28:45.911502 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de054b-15df-4a96-b150-ba32ff35c4a2 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:45.923066 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de054b-15df-4a96-b150-ba32ff35c4a2 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:45.927838 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e706bb-2c71-450a-a9a2-8ed6efc56df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:45.928528 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e706bb-2c71-450a-a9a2-8ed6efc56df0 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:45.928832 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e706bb-2c71-450a-a9a2-8ed6efc56df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:45.929471 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e706bb-2c71-450a-a9a2-8ed6efc56df0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:45.930211 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e706bb-2c71-450a-a9a2-8ed6efc56df0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:45.931859 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e706bb-2c71-450a-a9a2-8ed6efc56df0 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:45.947727 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1391/2773] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:28:45.952092 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb681a1a-c596-4ea6-bfcc-9112c72f2cd4 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:45.962753 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb681a1a-c596-4ea6-bfcc-9112c72f2cd4 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:45.970495 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ef2151-05ae-4be1-ba1e-fffa0ed5f245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:45.971183 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef2151-05ae-4be1-ba1e-fffa0ed5f245 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:45.971469 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef2151-05ae-4be1-ba1e-fffa0ed5f245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:45.971779 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef2151-05ae-4be1-ba1e-fffa0ed5f245 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:45.972521 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ef2151-05ae-4be1-ba1e-fffa0ed5f245 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:45.974131 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ef2151-05ae-4be1-ba1e-fffa0ed5f245 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:45.985137 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1383/2774] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:28:45.989245 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e706bb-2c71-450a-a9a2-8ed6efc56df0 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:46.000204 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e706bb-2c71-450a-a9a2-8ed6efc56df0 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:46.005039 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a0aff-cf1b-406d-b8b1-048bc2e2a3b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:46.005929 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a0aff-cf1b-406d-b8b1-048bc2e2a3b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:46.006213 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a0aff-cf1b-406d-b8b1-048bc2e2a3b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:46.006487 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a0aff-cf1b-406d-b8b1-048bc2e2a3b5 admin admin] PATH_INFO: `/v3/projects/720561fe8ccc404cb04b5edcd314b196/users/6b5763f4ea4b46ea91fe92dd60abccd5/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:46.024360 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3a0aff-cf1b-406d-b8b1-048bc2e2a3b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=720561fe8ccc404cb04b5edcd314b196, user_id=6b5763f4ea4b46ea91fe92dd60abccd5, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:46.027931 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3a0aff-cf1b-406d-b8b1-048bc2e2a3b5 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:46.041610 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1392/2775] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:28:45 2022] PUT /identity/v3/projects/720561fe8ccc404cb04b5edcd314b196/users/6b5763f4ea4b46ea91fe92dd60abccd5/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:46.052980 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd4d61-e772-49ae-aefc-46c8c1c35bde None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:46.053282 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd4d61-e772-49ae-aefc-46c8c1c35bde None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:46.053589 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd4d61-e772-49ae-aefc-46c8c1c35bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:46.084112 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-c1dd4d61-e772-49ae-aefc-46c8c1c35bde None None] MFA Rules not processed for user `6b5763f4ea4b46ea91fe92dd60abccd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:46.085349 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-c1dd4d61-e772-49ae-aefc-46c8c1c35bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:46.152366 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1384/2776] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:46.163478 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a0aff-cf1b-406d-b8b1-048bc2e2a3b5 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:46.165202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-ef3a0aff-cf1b-406d-b8b1-048bc2e2a3b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:46.202452 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:46.232680 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a0aff-cf1b-406d-b8b1-048bc2e2a3b5 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:46.237743 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1778f7c2-4992-4812-8fd0-43e7417fa472 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:46.238385 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778f7c2-4992-4812-8fd0-43e7417fa472 service cinder] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:46.238670 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778f7c2-4992-4812-8fd0-43e7417fa472 service cinder] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:46.238952 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778f7c2-4992-4812-8fd0-43e7417fa472 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:46.249244 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1778f7c2-4992-4812-8fd0-43e7417fa472 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:46.250954 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1778f7c2-4992-4812-8fd0-43e7417fa472 service cinder] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:46.260233 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1393/2777] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:28:46.279877 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd4d61-e772-49ae-aefc-46c8c1c35bde None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:46.327335 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:46.347086 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd4d61-e772-49ae-aefc-46c8c1c35bde None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:46.352110 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85b7a0-a645-4857-9c1d-7ada6dacd6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:46.352868 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85b7a0-a645-4857-9c1d-7ada6dacd6db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:46.353148 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85b7a0-a645-4857-9c1d-7ada6dacd6db admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:46.353419 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85b7a0-a645-4857-9c1d-7ada6dacd6db admin admin] PATH_INFO: `/v3/users/6b5763f4ea4b46ea91fe92dd60abccd5` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:46.354638 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb85b7a0-a645-4857-9c1d-7ada6dacd6db admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b5763f4ea4b46ea91fe92dd60abccd5)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:46.356632 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb85b7a0-a645-4857-9c1d-7ada6dacd6db admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:46.408367 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-eb85b7a0-a645-4857-9c1d-7ada6dacd6db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b5763f4ea4b46ea91fe92dd60abccd5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:46.409151 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-eb85b7a0-a645-4857-9c1d-7ada6dacd6db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b5763f4ea4b46ea91fe92dd60abccd5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:46.422327 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-eb85b7a0-a645-4857-9c1d-7ada6dacd6db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b5763f4ea4b46ea91fe92dd60abccd5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:46.429506 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-eb85b7a0-a645-4857-9c1d-7ada6dacd6db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b5763f4ea4b46ea91fe92dd60abccd5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:46.437073 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-eb85b7a0-a645-4857-9c1d-7ada6dacd6db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b5763f4ea4b46ea91fe92dd60abccd5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:46.443180 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1385/2778] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:28:46 2022] DELETE /identity/v3/users/6b5763f4ea4b46ea91fe92dd60abccd5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:46.448115 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f78f3e-93d3-4894-8433-0480fbb87d95 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:46.448686 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f78f3e-93d3-4894-8433-0480fbb87d95 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:46.448686 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f78f3e-93d3-4894-8433-0480fbb87d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:46.466196 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-c7f78f3e-93d3-4894-8433-0480fbb87d95 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:46.467417 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-c7f78f3e-93d3-4894-8433-0480fbb87d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:46.531718 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1394/2779] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:28:46.542635 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85b7a0-a645-4857-9c1d-7ada6dacd6db admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:46.545062 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-eb85b7a0-a645-4857-9c1d-7ada6dacd6db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:46.602818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85b7a0-a645-4857-9c1d-7ada6dacd6db admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:46.607168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f380f-cce6-4517-a736-457beca4fbd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:46.607845 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f380f-cce6-4517-a736-457beca4fbd6 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:46.608128 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f380f-cce6-4517-a736-457beca4fbd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:46.608400 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f380f-cce6-4517-a736-457beca4fbd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:46.617759 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185f380f-cce6-4517-a736-457beca4fbd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:46.619183 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185f380f-cce6-4517-a736-457beca4fbd6 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:46.629574 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1386/2780] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:28:46.921759 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f78f3e-93d3-4894-8433-0480fbb87d95 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:46.923285 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-c7f78f3e-93d3-4894-8433-0480fbb87d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:46.937969 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f78f3e-93d3-4894-8433-0480fbb87d95 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:46.943355 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f94812-3914-46dc-9a23-0d1794d69480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:46.944553 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f94812-3914-46dc-9a23-0d1794d69480 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:46.944835 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f94812-3914-46dc-9a23-0d1794d69480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:46.945111 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f94812-3914-46dc-9a23-0d1794d69480 admin admin] PATH_INFO: `/v3/projects/720561fe8ccc404cb04b5edcd314b196` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:46.946332 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f94812-3914-46dc-9a23-0d1794d69480 admin admin] RBAC: Authorizing `identity:delete_project(project_id=720561fe8ccc404cb04b5edcd314b196)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:46.948364 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f94812-3914-46dc-9a23-0d1794d69480 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:47.000368 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-13f94812-3914-46dc-9a23-0d1794d69480 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '720561fe8ccc404cb04b5edcd314b196', 'request_id': 'req-13f94812-3914-46dc-9a23-0d1794d69480'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:47.006945 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-13f94812-3914-46dc-9a23-0d1794d69480 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '720561fe8ccc404cb04b5edcd314b196', 'request_id': 'req-13f94812-3914-46dc-9a23-0d1794d69480'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:47.015768 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-13f94812-3914-46dc-9a23-0d1794d69480 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '720561fe8ccc404cb04b5edcd314b196', 'request_id': 'req-13f94812-3914-46dc-9a23-0d1794d69480'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:47.026343 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-13f94812-3914-46dc-9a23-0d1794d69480 admin admin] The token cache is being invalidate because project 720561fe8ccc404cb04b5edcd314b196 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:28:47.026636 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-13f94812-3914-46dc-9a23-0d1794d69480 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:47.027828 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-13f94812-3914-46dc-9a23-0d1794d69480 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:47.029425 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1395/2781] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:28:46 2022] DELETE /identity/v3/projects/720561fe8ccc404cb04b5edcd314b196 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:47.034405 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dccf2d0-8cae-42e1-9d25-de4d3ffa8bb0 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:47.034634 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dccf2d0-8cae-42e1-9d25-de4d3ffa8bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:47.034928 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dccf2d0-8cae-42e1-9d25-de4d3ffa8bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:47.053587 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-2dccf2d0-8cae-42e1-9d25-de4d3ffa8bb0 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:47.054954 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-2dccf2d0-8cae-42e1-9d25-de4d3ffa8bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:47.115855 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1387/2782] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:28:47.125056 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f736d-2c56-43a7-be7d-bd96fe8282b0 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:47.125056 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f736d-2c56-43a7-be7d-bd96fe8282b0 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:47.125294 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f736d-2c56-43a7-be7d-bd96fe8282b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:47.144016 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-f56f736d-2c56-43a7-be7d-bd96fe8282b0 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:47.145217 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-f56f736d-2c56-43a7-be7d-bd96fe8282b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:47.153691 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1396/2783] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:47.157895 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dccf2d0-8cae-42e1-9d25-de4d3ffa8bb0 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:47.164395 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:47.171575 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dccf2d0-8cae-42e1-9d25-de4d3ffa8bb0 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:47.177459 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23846e96-8148-433a-aee6-e9bc286f346f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:47.178315 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23846e96-8148-433a-aee6-e9bc286f346f admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:47.178772 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23846e96-8148-433a-aee6-e9bc286f346f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:47.179203 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23846e96-8148-433a-aee6-e9bc286f346f admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:47.180060 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23846e96-8148-433a-aee6-e9bc286f346f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:28:47.180498 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23846e96-8148-433a-aee6-e9bc286f346f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:47.182268 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23846e96-8148-433a-aee6-e9bc286f346f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:47.197839 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1388/2784] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:28:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:28:47.200565 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f736d-2c56-43a7-be7d-bd96fe8282b0 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:47.212206 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f736d-2c56-43a7-be7d-bd96fe8282b0 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:47.220184 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35839257-d4bb-43ba-a7b1-49da6325dfa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:47.220926 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35839257-d4bb-43ba-a7b1-49da6325dfa0 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:47.221175 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35839257-d4bb-43ba-a7b1-49da6325dfa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:47.221479 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35839257-d4bb-43ba-a7b1-49da6325dfa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:47.225448 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35839257-d4bb-43ba-a7b1-49da6325dfa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:47.225779 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35839257-d4bb-43ba-a7b1-49da6325dfa0 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:47.264404 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1397/2785] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:28:47 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:28:47.268931 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23846e96-8148-433a-aee6-e9bc286f346f admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:47.289914 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23846e96-8148-433a-aee6-e9bc286f346f admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:47.296235 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35839257-d4bb-43ba-a7b1-49da6325dfa0 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:47.298023 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-35839257-d4bb-43ba-a7b1-49da6325dfa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:47.330159 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:47.351747 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd618d3-8d2f-45fa-b6a3-a1b364609e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:47.352430 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd618d3-8d2f-45fa-b6a3-a1b364609e53 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:47.352714 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd618d3-8d2f-45fa-b6a3-a1b364609e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:47.352991 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd618d3-8d2f-45fa-b6a3-a1b364609e53 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:47.353685 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd618d3-8d2f-45fa-b6a3-a1b364609e53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:47.355772 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd618d3-8d2f-45fa-b6a3-a1b364609e53 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:47.373148 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35839257-d4bb-43ba-a7b1-49da6325dfa0 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:47.379024 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f31a3d4-415c-421c-a8a9-dff2e7d3ba2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:47.380017 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f31a3d4-415c-421c-a8a9-dff2e7d3ba2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:47.380301 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f31a3d4-415c-421c-a8a9-dff2e7d3ba2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:47.380582 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f31a3d4-415c-421c-a8a9-dff2e7d3ba2d admin admin] PATH_INFO: `/v3/users/c819429c1e2f4fa18abda4402277a125` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:47.381831 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f31a3d4-415c-421c-a8a9-dff2e7d3ba2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c819429c1e2f4fa18abda4402277a125)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:47.383704 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1389/2786] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:28:47 2022] POST /identity/v3/users => generated 293 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:28:47.383841 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f31a3d4-415c-421c-a8a9-dff2e7d3ba2d admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:47.388494 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd618d3-8d2f-45fa-b6a3-a1b364609e53 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:47.398679 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd618d3-8d2f-45fa-b6a3-a1b364609e53 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:47.404666 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a8cd0-fa1a-4e86-9ab1-3b3e4e677452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:47.405322 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a8cd0-fa1a-4e86-9ab1-3b3e4e677452 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:47.405606 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a8cd0-fa1a-4e86-9ab1-3b3e4e677452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:47.405886 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a8cd0-fa1a-4e86-9ab1-3b3e4e677452 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:47.406613 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a8cd0-fa1a-4e86-9ab1-3b3e4e677452 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:47.408254 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a8cd0-fa1a-4e86-9ab1-3b3e4e677452 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:47.423814 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1390/2787] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:28:47.424855 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a8cd0-fa1a-4e86-9ab1-3b3e4e677452 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:47.435140 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a8cd0-fa1a-4e86-9ab1-3b3e4e677452 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:47.436300 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-9f31a3d4-415c-421c-a8a9-dff2e7d3ba2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c819429c1e2f4fa18abda4402277a125'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:47.436976 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-9f31a3d4-415c-421c-a8a9-dff2e7d3ba2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c819429c1e2f4fa18abda4402277a125'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:47.439971 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3955ec58-a3bd-4f71-8383-ce53e433d298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:47.440641 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955ec58-a3bd-4f71-8383-ce53e433d298 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:47.440985 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955ec58-a3bd-4f71-8383-ce53e433d298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:47.441206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3955ec58-a3bd-4f71-8383-ce53e433d298 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:47.441955 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3955ec58-a3bd-4f71-8383-ce53e433d298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:47.443576 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3955ec58-a3bd-4f71-8383-ce53e433d298 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:47.450137 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-9f31a3d4-415c-421c-a8a9-dff2e7d3ba2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c819429c1e2f4fa18abda4402277a125'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:47.456045 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1391/2788] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:28:47.459424 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-9f31a3d4-415c-421c-a8a9-dff2e7d3ba2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c819429c1e2f4fa18abda4402277a125'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:47.459424 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-9f31a3d4-415c-421c-a8a9-dff2e7d3ba2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c819429c1e2f4fa18abda4402277a125'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:47.460342 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3955ec58-a3bd-4f71-8383-ce53e433d298 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:47.462096 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3955ec58-a3bd-4f71-8383-ce53e433d298 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:47.476776 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1398/2789] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:28:47 2022] DELETE /identity/v3/users/c819429c1e2f4fa18abda4402277a125 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:28:47.477104 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3955ec58-a3bd-4f71-8383-ce53e433d298 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:47.481305 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a101ed1-f14c-4abc-bc11-f132e2ed4d23 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:47.481771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a101ed1-f14c-4abc-bc11-f132e2ed4d23 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:47.482075 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a101ed1-f14c-4abc-bc11-f132e2ed4d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:47.501256 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-0a101ed1-f14c-4abc-bc11-f132e2ed4d23 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:47.502490 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-0a101ed1-f14c-4abc-bc11-f132e2ed4d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:47.546528 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d867fb4-b853-443e-a759-4b3443549af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:47.547334 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d867fb4-b853-443e-a759-4b3443549af3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:47.547734 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d867fb4-b853-443e-a759-4b3443549af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:47.547903 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d867fb4-b853-443e-a759-4b3443549af3 admin admin] PATH_INFO: `/v3/projects/48b8dac120114fe981c1d1d292f7c8fe/users/1f47d35fa1ae4681a6dc37e19a886ed3/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:47.563684 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d867fb4-b853-443e-a759-4b3443549af3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48b8dac120114fe981c1d1d292f7c8fe, user_id=1f47d35fa1ae4681a6dc37e19a886ed3, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:47.566341 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d867fb4-b853-443e-a759-4b3443549af3 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:47.571395 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1399/2790] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:28:47.584346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a101ed1-f14c-4abc-bc11-f132e2ed4d23 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:47.584346 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-0a101ed1-f14c-4abc-bc11-f132e2ed4d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:47.584727 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1392/2791] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:28:47 2022] PUT /identity/v3/projects/48b8dac120114fe981c1d1d292f7c8fe/users/1f47d35fa1ae4681a6dc37e19a886ed3/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:28:47.595428 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98806dd0-950e-4dc9-986c-0e8dd9c0c2c3 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:47.595761 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98806dd0-950e-4dc9-986c-0e8dd9c0c2c3 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:47.596068 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98806dd0-950e-4dc9-986c-0e8dd9c0c2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:47.626829 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-98806dd0-950e-4dc9-986c-0e8dd9c0c2c3 None None] MFA Rules not processed for user `1f47d35fa1ae4681a6dc37e19a886ed3`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:47.628092 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-98806dd0-950e-4dc9-986c-0e8dd9c0c2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:47.639874 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:47.649213 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a101ed1-f14c-4abc-bc11-f132e2ed4d23 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:47.659921 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:47.701034 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1393/2792] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:28:47.701472 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef8ce0d-2f1a-4ec9-a149-4172035bcfd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:47.702151 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8ce0d-2f1a-4ec9-a149-4172035bcfd6 service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:47.702435 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8ce0d-2f1a-4ec9-a149-4172035bcfd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:47.702713 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8ce0d-2f1a-4ec9-a149-4172035bcfd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:47.706303 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98806dd0-950e-4dc9-986c-0e8dd9c0c2c3 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:47.771844 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98806dd0-950e-4dc9-986c-0e8dd9c0c2c3 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:47.773349 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef8ce0d-2f1a-4ec9-a149-4172035bcfd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:47.775046 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef8ce0d-2f1a-4ec9-a149-4172035bcfd6 service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:47.776657 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd913d-e755-455c-833a-254083fa8ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:47.777311 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd913d-e755-455c-833a-254083fa8ff1 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:47.777601 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd913d-e755-455c-833a-254083fa8ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:47.777885 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd913d-e755-455c-833a-254083fa8ff1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:47.778575 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd913d-e755-455c-833a-254083fa8ff1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:47.780562 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd913d-e755-455c-833a-254083fa8ff1 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:47.786502 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1400/2793] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:28:47.810308 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1394/2794] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:28:47 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:28:47.814148 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef8ce0d-2f1a-4ec9-a149-4172035bcfd6 service neutron] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:47.823794 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef8ce0d-2f1a-4ec9-a149-4172035bcfd6 service neutron] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:47.894477 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7324012-a6b5-4ead-8544-37e2b576a664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:47.895175 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7324012-a6b5-4ead-8544-37e2b576a664 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:47.895438 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7324012-a6b5-4ead-8544-37e2b576a664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:47.895743 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7324012-a6b5-4ead-8544-37e2b576a664 admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:47.896436 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7324012-a6b5-4ead-8544-37e2b576a664 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:47.898387 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7324012-a6b5-4ead-8544-37e2b576a664 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:47.920375 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1401/2795] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:28:47 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:28:47.924738 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd913d-e755-455c-833a-254083fa8ff1 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:47.937009 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd913d-e755-455c-833a-254083fa8ff1 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:47.942425 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e0ea1-e236-4c76-86b2-ffad3be1eefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:47.943190 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e0ea1-e236-4c76-86b2-ffad3be1eefb admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:47.943570 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e0ea1-e236-4c76-86b2-ffad3be1eefb admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:47.943919 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e0ea1-e236-4c76-86b2-ffad3be1eefb admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:47.944765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e0ea1-e236-4c76-86b2-ffad3be1eefb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:47.946974 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e0ea1-e236-4c76-86b2-ffad3be1eefb admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:47.959594 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1395/2796] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:28:47.964214 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7324012-a6b5-4ead-8544-37e2b576a664 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:47.975797 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7324012-a6b5-4ead-8544-37e2b576a664 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:47.981814 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c12d1-bff6-4539-9b76-4ccc59051124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:47.982499 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c12d1-bff6-4539-9b76-4ccc59051124 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:47.982788 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c12d1-bff6-4539-9b76-4ccc59051124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:47.983065 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c12d1-bff6-4539-9b76-4ccc59051124 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:47.983835 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357c12d1-bff6-4539-9b76-4ccc59051124 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:47.985466 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357c12d1-bff6-4539-9b76-4ccc59051124 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:47.996743 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1402/2797] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:28:48.001078 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e0ea1-e236-4c76-86b2-ffad3be1eefb admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:48.011943 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e0ea1-e236-4c76-86b2-ffad3be1eefb admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:48.016847 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442606d-3397-4899-b989-bae38fb3b89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:48.017786 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442606d-3397-4899-b989-bae38fb3b89f admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:48.018067 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442606d-3397-4899-b989-bae38fb3b89f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:48.018341 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442606d-3397-4899-b989-bae38fb3b89f admin admin] PATH_INFO: `/v3/projects/f7787e2dc1944d21941a8b51a1bf0267/users/62398105e9a7458b9bb8475f72ef4265/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:48.033927 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6442606d-3397-4899-b989-bae38fb3b89f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7787e2dc1944d21941a8b51a1bf0267, user_id=62398105e9a7458b9bb8475f72ef4265, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:48.036617 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6442606d-3397-4899-b989-bae38fb3b89f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:48.053079 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1396/2798] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:28:48 2022] PUT /identity/v3/projects/f7787e2dc1944d21941a8b51a1bf0267/users/62398105e9a7458b9bb8475f72ef4265/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:48.064302 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177465d-2da5-4d24-87db-01424382e3a9 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:48.064596 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177465d-2da5-4d24-87db-01424382e3a9 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:48.064879 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177465d-2da5-4d24-87db-01424382e3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:48.087930 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442606d-3397-4899-b989-bae38fb3b89f admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:48.091014 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-6442606d-3397-4899-b989-bae38fb3b89f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:48.098334 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-0177465d-2da5-4d24-87db-01424382e3a9 None None] MFA Rules not processed for user `62398105e9a7458b9bb8475f72ef4265`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:48.099551 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-0177465d-2da5-4d24-87db-01424382e3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:48.123519 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:48.173496 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442606d-3397-4899-b989-bae38fb3b89f admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:48.173932 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1403/2799] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:48.178587 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0177465d-2da5-4d24-87db-01424382e3a9 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:48.178822 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09976bf4-4c94-40b0-bb10-4925c8b2d2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:48.179693 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09976bf4-4c94-40b0-bb10-4925c8b2d2b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:48.179986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09976bf4-4c94-40b0-bb10-4925c8b2d2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:48.180258 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09976bf4-4c94-40b0-bb10-4925c8b2d2b8 admin admin] PATH_INFO: `/v3/projects/9f97cb0b78d241828644d3722ec1377e` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:48.181483 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09976bf4-4c94-40b0-bb10-4925c8b2d2b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f97cb0b78d241828644d3722ec1377e)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:48.186227 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09976bf4-4c94-40b0-bb10-4925c8b2d2b8 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:48.190252 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0177465d-2da5-4d24-87db-01424382e3a9 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:48.194878 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a568a4-da7e-4675-b1d8-587002cc80c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:48.195548 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a568a4-da7e-4675-b1d8-587002cc80c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:48.195840 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a568a4-da7e-4675-b1d8-587002cc80c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:48.196111 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a568a4-da7e-4675-b1d8-587002cc80c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:48.196792 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a568a4-da7e-4675-b1d8-587002cc80c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:48.198696 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a568a4-da7e-4675-b1d8-587002cc80c0 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:48.211825 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:48.229414 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1404/2800] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:28:48 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:28:48.231085 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a568a4-da7e-4675-b1d8-587002cc80c0 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:48.238545 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-09976bf4-4c94-40b0-bb10-4925c8b2d2b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f97cb0b78d241828644d3722ec1377e', 'request_id': 'req-09976bf4-4c94-40b0-bb10-4925c8b2d2b8'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:48.245173 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-09976bf4-4c94-40b0-bb10-4925c8b2d2b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f97cb0b78d241828644d3722ec1377e', 'request_id': 'req-09976bf4-4c94-40b0-bb10-4925c8b2d2b8'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:48.254399 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-09976bf4-4c94-40b0-bb10-4925c8b2d2b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f97cb0b78d241828644d3722ec1377e', 'request_id': 'req-09976bf4-4c94-40b0-bb10-4925c8b2d2b8'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:48.268535 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-09976bf4-4c94-40b0-bb10-4925c8b2d2b8 admin admin] The token cache is being invalidate because project 9f97cb0b78d241828644d3722ec1377e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:28:48.268953 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-09976bf4-4c94-40b0-bb10-4925c8b2d2b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:48.269787 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-09976bf4-4c94-40b0-bb10-4925c8b2d2b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:48.271797 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1397/2801] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:28:48 2022] DELETE /identity/v3/projects/9f97cb0b78d241828644d3722ec1377e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:28:48.276631 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09976bf4-4c94-40b0-bb10-4925c8b2d2b8 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:48.278687 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-09976bf4-4c94-40b0-bb10-4925c8b2d2b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:48.310957 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a568a4-da7e-4675-b1d8-587002cc80c0 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:48.315815 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5464e470-5603-4ec1-b24a-5f9fbc832f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:48.316518 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464e470-5603-4ec1-b24a-5f9fbc832f7a admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:48.316804 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464e470-5603-4ec1-b24a-5f9fbc832f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:48.317082 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464e470-5603-4ec1-b24a-5f9fbc832f7a admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:48.317774 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5464e470-5603-4ec1-b24a-5f9fbc832f7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:48.319884 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5464e470-5603-4ec1-b24a-5f9fbc832f7a admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:48.344325 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1405/2802] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:28:48 2022] POST /identity/v3/users => generated 298 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:28:48.348108 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5464e470-5603-4ec1-b24a-5f9fbc832f7a admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:48.349867 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-5464e470-5603-4ec1-b24a-5f9fbc832f7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:48.360883 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09976bf4-4c94-40b0-bb10-4925c8b2d2b8 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:48.365928 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222056c4-b53a-4bc2-ae93-3b6bb8659d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:48.366804 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222056c4-b53a-4bc2-ae93-3b6bb8659d99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:48.367081 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222056c4-b53a-4bc2-ae93-3b6bb8659d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:48.367798 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222056c4-b53a-4bc2-ae93-3b6bb8659d99 admin admin] PATH_INFO: `/v3/users/06b52da53c224d97a8eb1f17064f46a6` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:48.368674 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222056c4-b53a-4bc2-ae93-3b6bb8659d99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06b52da53c224d97a8eb1f17064f46a6)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:48.370673 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222056c4-b53a-4bc2-ae93-3b6bb8659d99 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:48.371295 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5464e470-5603-4ec1-b24a-5f9fbc832f7a admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:48.376120 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643cb60-bc60-4e45-961d-2595cddbc468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:48.376791 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643cb60-bc60-4e45-961d-2595cddbc468 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:48.377076 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643cb60-bc60-4e45-961d-2595cddbc468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:48.377355 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643cb60-bc60-4e45-961d-2595cddbc468 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:48.378127 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0643cb60-bc60-4e45-961d-2595cddbc468 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:48.379772 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0643cb60-bc60-4e45-961d-2595cddbc468 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:48.392413 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1406/2803] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:28:48.396958 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643cb60-bc60-4e45-961d-2595cddbc468 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:48.409699 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643cb60-bc60-4e45-961d-2595cddbc468 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:48.415609 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8671ba9-6a7c-401c-a8b6-42d750302d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:48.416529 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8671ba9-6a7c-401c-a8b6-42d750302d53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:48.416863 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8671ba9-6a7c-401c-a8b6-42d750302d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:48.417149 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8671ba9-6a7c-401c-a8b6-42d750302d53 admin admin] PATH_INFO: `/v3/projects/0656dd8abcad457087eeaabe13cb5d62/users/3655248dfff241f088918f3239ae876f/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:48.423192 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-222056c4-b53a-4bc2-ae93-3b6bb8659d99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06b52da53c224d97a8eb1f17064f46a6'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:48.423980 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-222056c4-b53a-4bc2-ae93-3b6bb8659d99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06b52da53c224d97a8eb1f17064f46a6'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:48.434010 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8671ba9-6a7c-401c-a8b6-42d750302d53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0656dd8abcad457087eeaabe13cb5d62, user_id=3655248dfff241f088918f3239ae876f, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:48.436548 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-222056c4-b53a-4bc2-ae93-3b6bb8659d99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06b52da53c224d97a8eb1f17064f46a6'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:48.436918 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8671ba9-6a7c-401c-a8b6-42d750302d53 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:48.444008 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-222056c4-b53a-4bc2-ae93-3b6bb8659d99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06b52da53c224d97a8eb1f17064f46a6'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:48.451873 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-222056c4-b53a-4bc2-ae93-3b6bb8659d99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06b52da53c224d97a8eb1f17064f46a6'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:48.453856 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1407/2804] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:28:48 2022] PUT /identity/v3/projects/0656dd8abcad457087eeaabe13cb5d62/users/3655248dfff241f088918f3239ae876f/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:28:48.457742 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8671ba9-6a7c-401c-a8b6-42d750302d53 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:48.459223 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-c8671ba9-6a7c-401c-a8b6-42d750302d53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:48.461183 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1398/2805] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:28:48 2022] DELETE /identity/v3/users/06b52da53c224d97a8eb1f17064f46a6 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:48.516519 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:48.540105 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8671ba9-6a7c-401c-a8b6-42d750302d53 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:48.604492 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b3fb56-1f37-45e2-840d-7aa8cfeff3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:48.605170 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3fb56-1f37-45e2-840d-7aa8cfeff3e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:48.606006 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3fb56-1f37-45e2-840d-7aa8cfeff3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:48.606310 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3fb56-1f37-45e2-840d-7aa8cfeff3e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:48.607049 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b3fb56-1f37-45e2-840d-7aa8cfeff3e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:48.608748 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b3fb56-1f37-45e2-840d-7aa8cfeff3e1 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:48.620015 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1408/2806] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:28:48.625003 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222056c4-b53a-4bc2-ae93-3b6bb8659d99 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:48.638402 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222056c4-b53a-4bc2-ae93-3b6bb8659d99 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:48.644198 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646216d-7bcf-4c38-810d-739f2a7ca515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:48.644651 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646216d-7bcf-4c38-810d-739f2a7ca515 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:48.644941 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646216d-7bcf-4c38-810d-739f2a7ca515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:48.645218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646216d-7bcf-4c38-810d-739f2a7ca515 admin admin] PATH_INFO: `/v3/projects/0656dd8abcad457087eeaabe13cb5d62/users/3655248dfff241f088918f3239ae876f/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:48.647297 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9646216d-7bcf-4c38-810d-739f2a7ca515 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0656dd8abcad457087eeaabe13cb5d62, user_id=3655248dfff241f088918f3239ae876f, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:48.650424 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9646216d-7bcf-4c38-810d-739f2a7ca515 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:48.662276 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1399/2807] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:28:48 2022] PUT /identity/v3/projects/0656dd8abcad457087eeaabe13cb5d62/users/3655248dfff241f088918f3239ae876f/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:48.673956 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e4ce34-a34b-44a5-8963-93035388b0a0 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:48.674247 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e4ce34-a34b-44a5-8963-93035388b0a0 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:48.674527 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e4ce34-a34b-44a5-8963-93035388b0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:48.706891 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-35e4ce34-a34b-44a5-8963-93035388b0a0 None None] MFA Rules not processed for user `3655248dfff241f088918f3239ae876f`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:48.708144 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-35e4ce34-a34b-44a5-8963-93035388b0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:48.759989 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646216d-7bcf-4c38-810d-739f2a7ca515 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:48.762554 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-9646216d-7bcf-4c38-810d-739f2a7ca515 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:48.789928 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:48.801612 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1409/2808] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:48 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:48.812868 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e4ce34-a34b-44a5-8963-93035388b0a0 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:48.814432 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-35e4ce34-a34b-44a5-8963-93035388b0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:48.852171 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646216d-7bcf-4c38-810d-739f2a7ca515 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:48.861691 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7199bc9d-0172-400c-babf-fb358266a561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:48.862931 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199bc9d-0172-400c-babf-fb358266a561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:48.863213 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199bc9d-0172-400c-babf-fb358266a561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:48.863491 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199bc9d-0172-400c-babf-fb358266a561 admin admin] PATH_INFO: `/v3/projects/00f76bf8bca8441abcb522d398e28fb3` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:48.864765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7199bc9d-0172-400c-babf-fb358266a561 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00f76bf8bca8441abcb522d398e28fb3)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:48.866726 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7199bc9d-0172-400c-babf-fb358266a561 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:48.874680 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e4ce34-a34b-44a5-8963-93035388b0a0 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:48.878437 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6217043-8b84-4132-92ee-7e11574cb5a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:48.879085 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6217043-8b84-4132-92ee-7e11574cb5a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:48.879369 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6217043-8b84-4132-92ee-7e11574cb5a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:48.879679 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6217043-8b84-4132-92ee-7e11574cb5a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:48.881477 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-c6217043-8b84-4132-92ee-7e11574cb5a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:48.921924 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:48.928021 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7199bc9d-0172-400c-babf-fb358266a561 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00f76bf8bca8441abcb522d398e28fb3', 'request_id': 'req-7199bc9d-0172-400c-babf-fb358266a561'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:48.937881 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7199bc9d-0172-400c-babf-fb358266a561 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00f76bf8bca8441abcb522d398e28fb3', 'request_id': 'req-7199bc9d-0172-400c-babf-fb358266a561'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:48.946473 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7199bc9d-0172-400c-babf-fb358266a561 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00f76bf8bca8441abcb522d398e28fb3', 'request_id': 'req-7199bc9d-0172-400c-babf-fb358266a561'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:48.954641 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6217043-8b84-4132-92ee-7e11574cb5a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:48.955947 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6217043-8b84-4132-92ee-7e11574cb5a4 service cinder] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:48.959104 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7199bc9d-0172-400c-babf-fb358266a561 admin admin] The token cache is being invalidate because project 00f76bf8bca8441abcb522d398e28fb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:28:48.959489 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7199bc9d-0172-400c-babf-fb358266a561 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:48.960386 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7199bc9d-0172-400c-babf-fb358266a561 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:48.965807 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1400/2809] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:28:48 2022] DELETE /identity/v3/projects/00f76bf8bca8441abcb522d398e28fb3 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:28:48.966860 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1410/2810] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:28:48.967461 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70a2d2-5bba-492b-bedf-b43963f268a0 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:48.967894 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70a2d2-5bba-492b-bedf-b43963f268a0 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:48.968248 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70a2d2-5bba-492b-bedf-b43963f268a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:48.992028 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-5a70a2d2-5bba-492b-bedf-b43963f268a0 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:48.993349 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5a70a2d2-5bba-492b-bedf-b43963f268a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:49.078850 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1401/2811] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:49.087867 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1e5eb-cf36-4920-8371-a66fdada15a7 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:49.088188 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1e5eb-cf36-4920-8371-a66fdada15a7 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:49.088461 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1e5eb-cf36-4920-8371-a66fdada15a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:49.106879 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-b5a1e5eb-cf36-4920-8371-a66fdada15a7 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:49.108232 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-b5a1e5eb-cf36-4920-8371-a66fdada15a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:49.116381 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1411/2812] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:49.121551 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a70a2d2-5bba-492b-bedf-b43963f268a0 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:49.131814 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a70a2d2-5bba-492b-bedf-b43963f268a0 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:49.137523 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f328dcc-cd15-4827-b4d0-f5baf0646a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:49.138217 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f328dcc-cd15-4827-b4d0-f5baf0646a07 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:49.138496 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f328dcc-cd15-4827-b4d0-f5baf0646a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:49.138764 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f328dcc-cd15-4827-b4d0-f5baf0646a07 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:49.139489 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f328dcc-cd15-4827-b4d0-f5baf0646a07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:28:49.139823 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f328dcc-cd15-4827-b4d0-f5baf0646a07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:49.141476 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f328dcc-cd15-4827-b4d0-f5baf0646a07 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:49.155338 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1402/2813] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:28:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 23:28:49.161567 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1e5eb-cf36-4920-8371-a66fdada15a7 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:49.169784 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1e5eb-cf36-4920-8371-a66fdada15a7 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:49.174642 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab07fca2-8943-490d-ac4f-3869d8f2297f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:49.175292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab07fca2-8943-490d-ac4f-3869d8f2297f admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:49.175591 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab07fca2-8943-490d-ac4f-3869d8f2297f admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:49.175885 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab07fca2-8943-490d-ac4f-3869d8f2297f admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:49.176642 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab07fca2-8943-490d-ac4f-3869d8f2297f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:49.178651 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab07fca2-8943-490d-ac4f-3869d8f2297f admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:49.208981 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1412/2814] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:28:49 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:28:49.214207 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f328dcc-cd15-4827-b4d0-f5baf0646a07 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:49.227087 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f328dcc-cd15-4827-b4d0-f5baf0646a07 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:49.279369 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:49.318150 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de646164-b5ce-431a-8ff7-4a32a8237a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:49.318826 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de646164-b5ce-431a-8ff7-4a32a8237a09 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:49.319106 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de646164-b5ce-431a-8ff7-4a32a8237a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:49.319382 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de646164-b5ce-431a-8ff7-4a32a8237a09 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:49.320134 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de646164-b5ce-431a-8ff7-4a32a8237a09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:49.322058 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de646164-b5ce-431a-8ff7-4a32a8237a09 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:49.347317 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1403/2815] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:28:49 2022] POST /identity/v3/users => generated 288 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 23:28:49.351648 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab07fca2-8943-490d-ac4f-3869d8f2297f admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:49.366573 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab07fca2-8943-490d-ac4f-3869d8f2297f admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:49.372413 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be83cf0-75e0-4f2e-8434-aa89f4422ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:49.373106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be83cf0-75e0-4f2e-8434-aa89f4422ddf admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:49.373385 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be83cf0-75e0-4f2e-8434-aa89f4422ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:49.373654 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be83cf0-75e0-4f2e-8434-aa89f4422ddf admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:49.374377 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be83cf0-75e0-4f2e-8434-aa89f4422ddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:49.376014 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be83cf0-75e0-4f2e-8434-aa89f4422ddf admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:49.390262 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1413/2816] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:28:49.391717 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de646164-b5ce-431a-8ff7-4a32a8237a09 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:49.402030 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de646164-b5ce-431a-8ff7-4a32a8237a09 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:49.407365 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2b3e3b-56ed-48fa-aad0-e178b36e2b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:49.408045 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b3e3b-56ed-48fa-aad0-e178b36e2b45 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:49.408329 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b3e3b-56ed-48fa-aad0-e178b36e2b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:49.408606 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2b3e3b-56ed-48fa-aad0-e178b36e2b45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:49.409332 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2b3e3b-56ed-48fa-aad0-e178b36e2b45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:49.411185 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2b3e3b-56ed-48fa-aad0-e178b36e2b45 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:49.421752 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1404/2817] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:28:49.426077 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be83cf0-75e0-4f2e-8434-aa89f4422ddf admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:49.436145 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be83cf0-75e0-4f2e-8434-aa89f4422ddf admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:49.440826 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1732e1b7-395f-413d-a4c1-9a271290782a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:49.441700 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732e1b7-395f-413d-a4c1-9a271290782a admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:49.441978 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732e1b7-395f-413d-a4c1-9a271290782a admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:49.442257 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732e1b7-395f-413d-a4c1-9a271290782a admin admin] PATH_INFO: `/v3/projects/958e8e2a460d433c8276f37bd0aa2fe9/users/60783965056349f0978bfd6dc9f55435/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:49.457267 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1732e1b7-395f-413d-a4c1-9a271290782a admin admin] RBAC: Authorizing `identity:create_grant(project_id=958e8e2a460d433c8276f37bd0aa2fe9, user_id=60783965056349f0978bfd6dc9f55435, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:49.459964 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1732e1b7-395f-413d-a4c1-9a271290782a admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:49.473683 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1414/2818] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:28:49 2022] PUT /identity/v3/projects/958e8e2a460d433c8276f37bd0aa2fe9/users/60783965056349f0978bfd6dc9f55435/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:49.484255 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac92c5-0a38-4baf-b251-98485a93d2b9 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:49.484541 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac92c5-0a38-4baf-b251-98485a93d2b9 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:49.484843 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac92c5-0a38-4baf-b251-98485a93d2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:49.514190 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-38ac92c5-0a38-4baf-b251-98485a93d2b9 None None] MFA Rules not processed for user `60783965056349f0978bfd6dc9f55435`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:49.515431 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-38ac92c5-0a38-4baf-b251-98485a93d2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:49.583986 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1405/2819] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:49.588658 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1732e1b7-395f-413d-a4c1-9a271290782a admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:49.650564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1732e1b7-395f-413d-a4c1-9a271290782a admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:49.655914 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc8d27-49da-4046-8a7c-5d8a647e4175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:49.656585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc8d27-49da-4046-8a7c-5d8a647e4175 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:49.656866 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc8d27-49da-4046-8a7c-5d8a647e4175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:49.657136 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc8d27-49da-4046-8a7c-5d8a647e4175 admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:49.657971 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bc8d27-49da-4046-8a7c-5d8a647e4175 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:49.659916 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bc8d27-49da-4046-8a7c-5d8a647e4175 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:49.689697 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1415/2820] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:28:49 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:28:49.693810 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac92c5-0a38-4baf-b251-98485a93d2b9 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:49.705211 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac92c5-0a38-4baf-b251-98485a93d2b9 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:49.763221 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3cf118-8b02-4fe4-ad97-02eea5ef1480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:49.763929 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3cf118-8b02-4fe4-ad97-02eea5ef1480 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:49.764214 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3cf118-8b02-4fe4-ad97-02eea5ef1480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:49.764493 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3cf118-8b02-4fe4-ad97-02eea5ef1480 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:49.765186 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3cf118-8b02-4fe4-ad97-02eea5ef1480 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:49.767097 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3cf118-8b02-4fe4-ad97-02eea5ef1480 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:49.787432 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1406/2821] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:28:49 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:28:49.791498 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc8d27-49da-4046-8a7c-5d8a647e4175 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:49.803110 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc8d27-49da-4046-8a7c-5d8a647e4175 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:49.807868 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae432672-74f6-47ac-9163-7251215305ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:49.809016 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae432672-74f6-47ac-9163-7251215305ad admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:49.809322 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae432672-74f6-47ac-9163-7251215305ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:49.809681 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae432672-74f6-47ac-9163-7251215305ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:49.810330 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae432672-74f6-47ac-9163-7251215305ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:49.811974 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae432672-74f6-47ac-9163-7251215305ad admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:49.822619 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1416/2822] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:28:49.827910 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3cf118-8b02-4fe4-ad97-02eea5ef1480 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:49.838150 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3cf118-8b02-4fe4-ad97-02eea5ef1480 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:49.842741 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f42362-29d8-4517-b075-91fe16dbf2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:49.843389 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f42362-29d8-4517-b075-91fe16dbf2fd admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:49.843701 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f42362-29d8-4517-b075-91fe16dbf2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:49.843975 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f42362-29d8-4517-b075-91fe16dbf2fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:49.844694 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f42362-29d8-4517-b075-91fe16dbf2fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:49.846280 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f42362-29d8-4517-b075-91fe16dbf2fd admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:49.857987 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1407/2823] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:28:49.862204 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae432672-74f6-47ac-9163-7251215305ad admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:49.872871 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae432672-74f6-47ac-9163-7251215305ad admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:49.877451 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc27f8-eb4e-4870-99b9-24fe6eb6284a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:49.878347 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc27f8-eb4e-4870-99b9-24fe6eb6284a admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:49.878775 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc27f8-eb4e-4870-99b9-24fe6eb6284a admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:49.879239 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc27f8-eb4e-4870-99b9-24fe6eb6284a admin admin] PATH_INFO: `/v3/projects/bcff634b060a4b0e92725eeac37a95ea/users/f35d753aedba4fae886287715008433c/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:49.894610 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edc27f8-eb4e-4870-99b9-24fe6eb6284a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcff634b060a4b0e92725eeac37a95ea, user_id=f35d753aedba4fae886287715008433c, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:49.897307 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edc27f8-eb4e-4870-99b9-24fe6eb6284a admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:49.910660 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1417/2824] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:28:49 2022] PUT /identity/v3/projects/bcff634b060a4b0e92725eeac37a95ea/users/f35d753aedba4fae886287715008433c/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:49.921830 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36234e-0269-4e2c-acdd-e8402cf7c2d4 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:49.922150 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36234e-0269-4e2c-acdd-e8402cf7c2d4 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:49.922568 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36234e-0269-4e2c-acdd-e8402cf7c2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:49.955727 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-3d36234e-0269-4e2c-acdd-e8402cf7c2d4 None None] MFA Rules not processed for user `f35d753aedba4fae886287715008433c`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:49.956955 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3d36234e-0269-4e2c-acdd-e8402cf7c2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:49.967170 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:50.024271 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1408/2825] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:50.028875 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc27f8-eb4e-4870-99b9-24fe6eb6284a admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:50.053953 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:50.092458 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc27f8-eb4e-4870-99b9-24fe6eb6284a admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:50.097213 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f074fd-c1a4-4ab1-aa21-53cef3e8e93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:50.097914 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f074fd-c1a4-4ab1-aa21-53cef3e8e93e admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:50.098201 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f074fd-c1a4-4ab1-aa21-53cef3e8e93e admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:50.098481 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f074fd-c1a4-4ab1-aa21-53cef3e8e93e admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:50.099189 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f074fd-c1a4-4ab1-aa21-53cef3e8e93e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:50.101151 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f074fd-c1a4-4ab1-aa21-53cef3e8e93e admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:50.126647 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1418/2826] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:28:50 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:28:50.130836 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d36234e-0269-4e2c-acdd-e8402cf7c2d4 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:50.142109 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d36234e-0269-4e2c-acdd-e8402cf7c2d4 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:50.197712 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284793d4-ad9a-4a64-8e65-808a05d7a4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:50.198291 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284793d4-ad9a-4a64-8e65-808a05d7a4ca admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:50.198571 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284793d4-ad9a-4a64-8e65-808a05d7a4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:50.198846 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284793d4-ad9a-4a64-8e65-808a05d7a4ca admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:50.199518 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284793d4-ad9a-4a64-8e65-808a05d7a4ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:50.201457 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284793d4-ad9a-4a64-8e65-808a05d7a4ca admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:50.220656 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1409/2827] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:28:50 2022] POST /identity/v3/users => generated 295 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:28:50.224771 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f074fd-c1a4-4ab1-aa21-53cef3e8e93e admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:50.236614 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f074fd-c1a4-4ab1-aa21-53cef3e8e93e admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:50.241577 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473fb01-05c3-4229-bd34-147147314954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:50.242368 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473fb01-05c3-4229-bd34-147147314954 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:50.242655 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473fb01-05c3-4229-bd34-147147314954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:50.242928 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473fb01-05c3-4229-bd34-147147314954 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:50.243868 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e473fb01-05c3-4229-bd34-147147314954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:50.245401 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e473fb01-05c3-4229-bd34-147147314954 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:50.259409 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1419/2828] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:28:50.264079 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284793d4-ad9a-4a64-8e65-808a05d7a4ca admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:50.274574 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284793d4-ad9a-4a64-8e65-808a05d7a4ca admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:50.279346 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e92ac-7aa7-44d3-8acb-860b48016b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:50.280331 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e92ac-7aa7-44d3-8acb-860b48016b88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:50.280641 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e92ac-7aa7-44d3-8acb-860b48016b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:50.280918 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e92ac-7aa7-44d3-8acb-860b48016b88 admin admin] PATH_INFO: `/v3/projects/47d9afb8dc9344ac937e2af8cdbac461/users/2408c2a7c8f947fb9686a5b5368fd9c6/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:50.297756 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e92ac-7aa7-44d3-8acb-860b48016b88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47d9afb8dc9344ac937e2af8cdbac461, user_id=2408c2a7c8f947fb9686a5b5368fd9c6, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:50.300522 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e92ac-7aa7-44d3-8acb-860b48016b88 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:50.314252 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1410/2829] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:28:50 2022] PUT /identity/v3/projects/47d9afb8dc9344ac937e2af8cdbac461/users/2408c2a7c8f947fb9686a5b5368fd9c6/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:28:50.318378 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473fb01-05c3-4229-bd34-147147314954 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:50.328671 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e473fb01-05c3-4229-bd34-147147314954 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:50.390957 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c17e7b4-9d76-4819-842c-f34de3ad2d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:50.392582 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c17e7b4-9d76-4819-842c-f34de3ad2d44 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:50.392977 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c17e7b4-9d76-4819-842c-f34de3ad2d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:50.393315 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c17e7b4-9d76-4819-842c-f34de3ad2d44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:50.394322 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c17e7b4-9d76-4819-842c-f34de3ad2d44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:50.396451 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c17e7b4-9d76-4819-842c-f34de3ad2d44 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:50.412962 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1420/2830] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:28:50.419635 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e92ac-7aa7-44d3-8acb-860b48016b88 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:50.430775 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e92ac-7aa7-44d3-8acb-860b48016b88 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:50.436969 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b85ae7-6746-4d72-890e-3d8a8d6ff0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:50.438014 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b85ae7-6746-4d72-890e-3d8a8d6ff0fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:50.438501 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b85ae7-6746-4d72-890e-3d8a8d6ff0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:50.438807 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b85ae7-6746-4d72-890e-3d8a8d6ff0fe admin admin] PATH_INFO: `/v3/projects/47d9afb8dc9344ac937e2af8cdbac461/users/2408c2a7c8f947fb9686a5b5368fd9c6/roles/8cfd0b6e71e14e0983c3eb4d7478e027` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:50.441037 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b85ae7-6746-4d72-890e-3d8a8d6ff0fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=47d9afb8dc9344ac937e2af8cdbac461, user_id=2408c2a7c8f947fb9686a5b5368fd9c6, role_id=8cfd0b6e71e14e0983c3eb4d7478e027)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:50.443912 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b85ae7-6746-4d72-890e-3d8a8d6ff0fe admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:50.460124 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1411/2831] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:28:50 2022] PUT /identity/v3/projects/47d9afb8dc9344ac937e2af8cdbac461/users/2408c2a7c8f947fb9686a5b5368fd9c6/roles/8cfd0b6e71e14e0983c3eb4d7478e027 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:28:50.524935 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c17e7b4-9d76-4819-842c-f34de3ad2d44 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:50.526972 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-5c17e7b4-9d76-4819-842c-f34de3ad2d44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:50.528765 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647afa36-b05f-4b0a-8955-2ef63592ab82 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:50.529048 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647afa36-b05f-4b0a-8955-2ef63592ab82 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:50.529327 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647afa36-b05f-4b0a-8955-2ef63592ab82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:50.558177 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-647afa36-b05f-4b0a-8955-2ef63592ab82 None None] MFA Rules not processed for user `2408c2a7c8f947fb9686a5b5368fd9c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:50.559438 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-647afa36-b05f-4b0a-8955-2ef63592ab82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:50.587417 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c17e7b4-9d76-4819-842c-f34de3ad2d44 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:50.613849 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:50.634181 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:50.635105 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1412/2832] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:50 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:28:50.641321 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571e793-41c1-4283-9c14-20c1cd574186 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:50.641940 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571e793-41c1-4283-9c14-20c1cd574186 service cinder] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:50.642423 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571e793-41c1-4283-9c14-20c1cd574186 service cinder] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:50.642893 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571e793-41c1-4283-9c14-20c1cd574186 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:50.644726 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-5571e793-41c1-4283-9c14-20c1cd574186 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:50.647168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647afa36-b05f-4b0a-8955-2ef63592ab82 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:50.648560 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-647afa36-b05f-4b0a-8955-2ef63592ab82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:50.661089 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647afa36-b05f-4b0a-8955-2ef63592ab82 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:50.664870 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11871000-19b7-40dc-8ef8-09167f0fc8f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:50.665504 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11871000-19b7-40dc-8ef8-09167f0fc8f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:50.665784 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11871000-19b7-40dc-8ef8-09167f0fc8f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:50.666057 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11871000-19b7-40dc-8ef8-09167f0fc8f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:50.708637 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5571e793-41c1-4283-9c14-20c1cd574186 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:50.710037 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5571e793-41c1-4283-9c14-20c1cd574186 service cinder] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:50.719916 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1421/2833] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:28:50.728182 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11871000-19b7-40dc-8ef8-09167f0fc8f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:50.729633 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11871000-19b7-40dc-8ef8-09167f0fc8f2 service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:50.739208 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1413/2834] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:28:51.276396 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571e793-41c1-4283-9c14-20c1cd574186 service cinder] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:51.286257 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571e793-41c1-4283-9c14-20c1cd574186 service cinder] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:51.290576 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c35fb5d-5cfe-42ad-9eba-022602ecda1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:51.291222 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35fb5d-5cfe-42ad-9eba-022602ecda1d service cinder] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:51.291556 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35fb5d-5cfe-42ad-9eba-022602ecda1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:51.291921 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35fb5d-5cfe-42ad-9eba-022602ecda1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:51.293986 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-0c35fb5d-5cfe-42ad-9eba-022602ecda1d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:51.366615 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c35fb5d-5cfe-42ad-9eba-022602ecda1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:51.368915 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c35fb5d-5cfe-42ad-9eba-022602ecda1d service cinder] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:51.386246 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1422/2835] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:51 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:28:52.244391 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11871000-19b7-40dc-8ef8-09167f0fc8f2 service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:52.253149 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11871000-19b7-40dc-8ef8-09167f0fc8f2 service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:52.258503 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ab9124-2b20-4828-b5ab-e7d51d3907ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:52.259454 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab9124-2b20-4828-b5ab-e7d51d3907ba service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:52.259825 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab9124-2b20-4828-b5ab-e7d51d3907ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:52.260170 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab9124-2b20-4828-b5ab-e7d51d3907ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:52.322072 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ab9124-2b20-4828-b5ab-e7d51d3907ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:52.323409 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ab9124-2b20-4828-b5ab-e7d51d3907ba service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:52.333014 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1414/2836] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:28:52.908613 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c35fb5d-5cfe-42ad-9eba-022602ecda1d service cinder] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:52.918011 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c35fb5d-5cfe-42ad-9eba-022602ecda1d service cinder] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:52.923884 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7292e9-0c7c-4226-805b-21a9d2a9b4f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:52.924889 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7292e9-0c7c-4226-805b-21a9d2a9b4f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:52.925269 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7292e9-0c7c-4226-805b-21a9d2a9b4f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:52.925636 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7292e9-0c7c-4226-805b-21a9d2a9b4f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:52.936663 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7292e9-0c7c-4226-805b-21a9d2a9b4f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:52.938290 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7292e9-0c7c-4226-805b-21a9d2a9b4f6 service cinder] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:52.950271 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1423/2837] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:52 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:28:55.409988 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ab9124-2b20-4828-b5ab-e7d51d3907ba service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:55.411489 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-20ab9124-2b20-4828-b5ab-e7d51d3907ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:55.476194 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ab9124-2b20-4828-b5ab-e7d51d3907ba service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:55.480805 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20625f54-2b9a-4204-ad66-cabc9bd4b054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:55.481668 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20625f54-2b9a-4204-ad66-cabc9bd4b054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:55.481959 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20625f54-2b9a-4204-ad66-cabc9bd4b054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:55.482223 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20625f54-2b9a-4204-ad66-cabc9bd4b054 admin admin] PATH_INFO: `/v3/users/1f47d35fa1ae4681a6dc37e19a886ed3` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:55.483431 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20625f54-2b9a-4204-ad66-cabc9bd4b054 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f47d35fa1ae4681a6dc37e19a886ed3)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:55.485392 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20625f54-2b9a-4204-ad66-cabc9bd4b054 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:55.514209 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:55.533535 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-20625f54-2b9a-4204-ad66-cabc9bd4b054 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f47d35fa1ae4681a6dc37e19a886ed3'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:55.534187 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-20625f54-2b9a-4204-ad66-cabc9bd4b054 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f47d35fa1ae4681a6dc37e19a886ed3'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:55.547474 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-20625f54-2b9a-4204-ad66-cabc9bd4b054 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f47d35fa1ae4681a6dc37e19a886ed3'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:55.556607 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-20625f54-2b9a-4204-ad66-cabc9bd4b054 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f47d35fa1ae4681a6dc37e19a886ed3'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:55.563777 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-20625f54-2b9a-4204-ad66-cabc9bd4b054 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f47d35fa1ae4681a6dc37e19a886ed3'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:55.569849 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1415/2838] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:28:55 2022] DELETE /identity/v3/users/1f47d35fa1ae4681a6dc37e19a886ed3 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:55.575123 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badce860-c4cc-4ca9-a678-0b20e80ccb6f None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:55.575674 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badce860-c4cc-4ca9-a678-0b20e80ccb6f None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:55.575808 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badce860-c4cc-4ca9-a678-0b20e80ccb6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:55.593796 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-badce860-c4cc-4ca9-a678-0b20e80ccb6f None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:55.594915 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-badce860-c4cc-4ca9-a678-0b20e80ccb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:55.657061 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1424/2839] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:28:55.667330 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20625f54-2b9a-4204-ad66-cabc9bd4b054 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:55.669317 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-20625f54-2b9a-4204-ad66-cabc9bd4b054 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:55.735642 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20625f54-2b9a-4204-ad66-cabc9bd4b054 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:55.739449 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e715d-4eca-4103-9bd8-610a47615525 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:55.740136 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e715d-4eca-4103-9bd8-610a47615525 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:55.740434 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e715d-4eca-4103-9bd8-610a47615525 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:55.740713 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e715d-4eca-4103-9bd8-610a47615525 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:55.750795 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93e715d-4eca-4103-9bd8-610a47615525 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:55.752281 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93e715d-4eca-4103-9bd8-610a47615525 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:55.763359 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1416/2840] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:28:56.038970 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badce860-c4cc-4ca9-a678-0b20e80ccb6f None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:56.040513 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-badce860-c4cc-4ca9-a678-0b20e80ccb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:56.053571 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badce860-c4cc-4ca9-a678-0b20e80ccb6f None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:56.058898 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a37176-0a6f-49ab-9540-e5b71db240e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:56.059841 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a37176-0a6f-49ab-9540-e5b71db240e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:56.060091 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a37176-0a6f-49ab-9540-e5b71db240e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:56.060428 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a37176-0a6f-49ab-9540-e5b71db240e3 admin admin] PATH_INFO: `/v3/projects/48b8dac120114fe981c1d1d292f7c8fe` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:56.061684 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a37176-0a6f-49ab-9540-e5b71db240e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48b8dac120114fe981c1d1d292f7c8fe)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:56.063653 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a37176-0a6f-49ab-9540-e5b71db240e3 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:56.086523 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=100 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:56.112167 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e6a37176-0a6f-49ab-9540-e5b71db240e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48b8dac120114fe981c1d1d292f7c8fe', 'request_id': 'req-e6a37176-0a6f-49ab-9540-e5b71db240e3'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:56.118420 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e6a37176-0a6f-49ab-9540-e5b71db240e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48b8dac120114fe981c1d1d292f7c8fe', 'request_id': 'req-e6a37176-0a6f-49ab-9540-e5b71db240e3'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:56.126271 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e6a37176-0a6f-49ab-9540-e5b71db240e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48b8dac120114fe981c1d1d292f7c8fe', 'request_id': 'req-e6a37176-0a6f-49ab-9540-e5b71db240e3'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:56.137296 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e6a37176-0a6f-49ab-9540-e5b71db240e3 admin admin] The token cache is being invalidate because project 48b8dac120114fe981c1d1d292f7c8fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:28:56.137582 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e6a37176-0a6f-49ab-9540-e5b71db240e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:56.138220 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e6a37176-0a6f-49ab-9540-e5b71db240e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:56.139795 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1425/2841] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:28:56 2022] DELETE /identity/v3/projects/48b8dac120114fe981c1d1d292f7c8fe => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:56.143752 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e715d-4eca-4103-9bd8-610a47615525 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:56.153306 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e715d-4eca-4103-9bd8-610a47615525 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:56.208203 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e280b0c4-95a7-4827-b8cb-4d830b53c49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:56.209063 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280b0c4-95a7-4827-b8cb-4d830b53c49b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:56.209339 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280b0c4-95a7-4827-b8cb-4d830b53c49b admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:56.209609 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280b0c4-95a7-4827-b8cb-4d830b53c49b admin admin] PATH_INFO: `/v3/users/62398105e9a7458b9bb8475f72ef4265` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:56.210778 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e280b0c4-95a7-4827-b8cb-4d830b53c49b admin admin] RBAC: Authorizing `identity:delete_user(user_id=62398105e9a7458b9bb8475f72ef4265)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:56.212705 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e280b0c4-95a7-4827-b8cb-4d830b53c49b admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:56.258015 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e280b0c4-95a7-4827-b8cb-4d830b53c49b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62398105e9a7458b9bb8475f72ef4265'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:56.258663 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e280b0c4-95a7-4827-b8cb-4d830b53c49b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62398105e9a7458b9bb8475f72ef4265'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:56.260534 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:56.272311 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e280b0c4-95a7-4827-b8cb-4d830b53c49b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62398105e9a7458b9bb8475f72ef4265'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:56.278938 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e280b0c4-95a7-4827-b8cb-4d830b53c49b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62398105e9a7458b9bb8475f72ef4265'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:56.286902 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e280b0c4-95a7-4827-b8cb-4d830b53c49b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62398105e9a7458b9bb8475f72ef4265'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:56.293519 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1417/2842] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:28:56 2022] DELETE /identity/v3/users/62398105e9a7458b9bb8475f72ef4265 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:56.582578 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a37176-0a6f-49ab-9540-e5b71db240e3 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:56.590775 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-e6a37176-0a6f-49ab-9540-e5b71db240e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:56.598315 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a37176-0a6f-49ab-9540-e5b71db240e3 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:56.665072 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c8507-1179-4253-909d-e2db46fd86d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:56.665992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c8507-1179-4253-909d-e2db46fd86d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:56.666259 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c8507-1179-4253-909d-e2db46fd86d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:56.666530 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c8507-1179-4253-909d-e2db46fd86d8 admin admin] PATH_INFO: `/v3/projects/f7787e2dc1944d21941a8b51a1bf0267` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:56.668622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c8507-1179-4253-909d-e2db46fd86d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7787e2dc1944d21941a8b51a1bf0267)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:56.670544 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c8507-1179-4253-909d-e2db46fd86d8 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:56.721158 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-1a4c8507-1179-4253-909d-e2db46fd86d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7787e2dc1944d21941a8b51a1bf0267', 'request_id': 'req-1a4c8507-1179-4253-909d-e2db46fd86d8'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:56.727954 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-1a4c8507-1179-4253-909d-e2db46fd86d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7787e2dc1944d21941a8b51a1bf0267', 'request_id': 'req-1a4c8507-1179-4253-909d-e2db46fd86d8'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:56.735855 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-1a4c8507-1179-4253-909d-e2db46fd86d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7787e2dc1944d21941a8b51a1bf0267', 'request_id': 'req-1a4c8507-1179-4253-909d-e2db46fd86d8'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:56.746739 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-1a4c8507-1179-4253-909d-e2db46fd86d8 admin admin] The token cache is being invalidate because project f7787e2dc1944d21941a8b51a1bf0267 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:28:56.747034 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-1a4c8507-1179-4253-909d-e2db46fd86d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:56.747992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-1a4c8507-1179-4253-909d-e2db46fd86d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:56.749822 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1426/2843] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:28:56 2022] DELETE /identity/v3/projects/f7787e2dc1944d21941a8b51a1bf0267 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:28:56.753835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e280b0c4-95a7-4827-b8cb-4d830b53c49b admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:56.755325 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e280b0c4-95a7-4827-b8cb-4d830b53c49b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:56.769896 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e280b0c4-95a7-4827-b8cb-4d830b53c49b admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:56.842890 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d2bef1-fd37-4933-b1c6-bfbaa2985300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:56.843802 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2bef1-fd37-4933-b1c6-bfbaa2985300 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:56.844088 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2bef1-fd37-4933-b1c6-bfbaa2985300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:56.844364 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2bef1-fd37-4933-b1c6-bfbaa2985300 admin admin] PATH_INFO: `/v3/users/3655248dfff241f088918f3239ae876f` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:56.845566 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d2bef1-fd37-4933-b1c6-bfbaa2985300 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3655248dfff241f088918f3239ae876f)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:56.847470 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d2bef1-fd37-4933-b1c6-bfbaa2985300 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:56.894131 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c0d2bef1-fd37-4933-b1c6-bfbaa2985300 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3655248dfff241f088918f3239ae876f'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:56.894812 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c0d2bef1-fd37-4933-b1c6-bfbaa2985300 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3655248dfff241f088918f3239ae876f'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:56.907511 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c0d2bef1-fd37-4933-b1c6-bfbaa2985300 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3655248dfff241f088918f3239ae876f'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:56.914249 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c0d2bef1-fd37-4933-b1c6-bfbaa2985300 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3655248dfff241f088918f3239ae876f'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:56.922168 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c0d2bef1-fd37-4933-b1c6-bfbaa2985300 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3655248dfff241f088918f3239ae876f'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:56.928390 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1418/2844] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:28:56 2022] DELETE /identity/v3/users/3655248dfff241f088918f3239ae876f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:57.203669 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c8507-1179-4253-909d-e2db46fd86d8 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:57.205135 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-1a4c8507-1179-4253-909d-e2db46fd86d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:57.218035 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c8507-1179-4253-909d-e2db46fd86d8 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:57.254742 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:57.279982 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da510bd0-3e76-40c8-93f9-d4b55bfd98b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:57.280845 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da510bd0-3e76-40c8-93f9-d4b55bfd98b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:57.281121 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da510bd0-3e76-40c8-93f9-d4b55bfd98b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:57.281391 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da510bd0-3e76-40c8-93f9-d4b55bfd98b9 admin admin] PATH_INFO: `/v3/projects/0656dd8abcad457087eeaabe13cb5d62` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:57.282575 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da510bd0-3e76-40c8-93f9-d4b55bfd98b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0656dd8abcad457087eeaabe13cb5d62)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:57.284521 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da510bd0-3e76-40c8-93f9-d4b55bfd98b9 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:57.329179 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-da510bd0-3e76-40c8-93f9-d4b55bfd98b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0656dd8abcad457087eeaabe13cb5d62', 'request_id': 'req-da510bd0-3e76-40c8-93f9-d4b55bfd98b9'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:57.335298 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-da510bd0-3e76-40c8-93f9-d4b55bfd98b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0656dd8abcad457087eeaabe13cb5d62', 'request_id': 'req-da510bd0-3e76-40c8-93f9-d4b55bfd98b9'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:57.343115 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-da510bd0-3e76-40c8-93f9-d4b55bfd98b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0656dd8abcad457087eeaabe13cb5d62', 'request_id': 'req-da510bd0-3e76-40c8-93f9-d4b55bfd98b9'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:57.353328 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-da510bd0-3e76-40c8-93f9-d4b55bfd98b9 admin admin] The token cache is being invalidate because project 0656dd8abcad457087eeaabe13cb5d62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:28:57.353619 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-da510bd0-3e76-40c8-93f9-d4b55bfd98b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:57.354289 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-da510bd0-3e76-40c8-93f9-d4b55bfd98b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:57.356551 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1427/2845] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:28:57 2022] DELETE /identity/v3/projects/0656dd8abcad457087eeaabe13cb5d62 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:57.361600 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d719e-ce6f-4ee0-9627-bd57c2025915 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:57.361893 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d719e-ce6f-4ee0-9627-bd57c2025915 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:57.362173 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d719e-ce6f-4ee0-9627-bd57c2025915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:57.380378 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-d32d719e-ce6f-4ee0-9627-bd57c2025915 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:57.381600 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-d32d719e-ce6f-4ee0-9627-bd57c2025915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:57.424498 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:57.443384 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1419/2846] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:57.451143 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2750347-92da-4ea8-a94d-1eb3e3b02e81 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:57.451432 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2750347-92da-4ea8-a94d-1eb3e3b02e81 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:57.451750 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2750347-92da-4ea8-a94d-1eb3e3b02e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:57.470421 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-b2750347-92da-4ea8-a94d-1eb3e3b02e81 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:57.471640 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-b2750347-92da-4ea8-a94d-1eb3e3b02e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:57.479435 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1428/2847] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:57.484128 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d719e-ce6f-4ee0-9627-bd57c2025915 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:57.493982 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d719e-ce6f-4ee0-9627-bd57c2025915 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:57.499148 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d288750f-5b8d-4321-9e58-09984d206d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:57.499865 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d288750f-5b8d-4321-9e58-09984d206d4e admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:57.500149 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d288750f-5b8d-4321-9e58-09984d206d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:57.500424 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d288750f-5b8d-4321-9e58-09984d206d4e admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:57.501216 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d288750f-5b8d-4321-9e58-09984d206d4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:28:57.501521 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d288750f-5b8d-4321-9e58-09984d206d4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:57.503184 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d288750f-5b8d-4321-9e58-09984d206d4e admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:57.516597 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1420/2848] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:28:57.521010 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2750347-92da-4ea8-a94d-1eb3e3b02e81 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:57.531292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2750347-92da-4ea8-a94d-1eb3e3b02e81 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:57.536154 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da7d0e-d2a4-4f45-83c1-dd046fb37471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:57.536824 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da7d0e-d2a4-4f45-83c1-dd046fb37471 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:57.537106 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da7d0e-d2a4-4f45-83c1-dd046fb37471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:57.537384 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da7d0e-d2a4-4f45-83c1-dd046fb37471 admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:57.538098 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2da7d0e-d2a4-4f45-83c1-dd046fb37471 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:57.540098 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2da7d0e-d2a4-4f45-83c1-dd046fb37471 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:57.568830 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1429/2849] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:28:57 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:28:57.573305 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d288750f-5b8d-4321-9e58-09984d206d4e admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:57.583647 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d288750f-5b8d-4321-9e58-09984d206d4e admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:57.646215 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbbcc36-d2e2-470e-aae7-e4b4fbfa9896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:57.646896 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbbcc36-d2e2-470e-aae7-e4b4fbfa9896 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:57.647246 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbbcc36-d2e2-470e-aae7-e4b4fbfa9896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:57.647496 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbbcc36-d2e2-470e-aae7-e4b4fbfa9896 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:57.648244 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbbcc36-d2e2-470e-aae7-e4b4fbfa9896 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:57.650216 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbbcc36-d2e2-470e-aae7-e4b4fbfa9896 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:57.671017 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1421/2850] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:28:57 2022] POST /identity/v3/users => generated 285 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:28:57.674951 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da7d0e-d2a4-4f45-83c1-dd046fb37471 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:57.685409 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da7d0e-d2a4-4f45-83c1-dd046fb37471 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:57.690087 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ffbf78-fa87-4288-99b9-96302b5b0acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:57.690737 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffbf78-fa87-4288-99b9-96302b5b0acb admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:57.691023 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffbf78-fa87-4288-99b9-96302b5b0acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:57.691302 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffbf78-fa87-4288-99b9-96302b5b0acb admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:57.692077 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ffbf78-fa87-4288-99b9-96302b5b0acb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:57.693679 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ffbf78-fa87-4288-99b9-96302b5b0acb admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:57.705811 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1430/2851] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:28:57.709715 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbbcc36-d2e2-470e-aae7-e4b4fbfa9896 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:57.720266 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbbcc36-d2e2-470e-aae7-e4b4fbfa9896 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:57.725160 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cabdd5-e8a8-4d91-a393-31a601ff6153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:57.725853 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cabdd5-e8a8-4d91-a393-31a601ff6153 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:57.726137 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cabdd5-e8a8-4d91-a393-31a601ff6153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:57.726410 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cabdd5-e8a8-4d91-a393-31a601ff6153 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:57.727134 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cabdd5-e8a8-4d91-a393-31a601ff6153 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:57.728808 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cabdd5-e8a8-4d91-a393-31a601ff6153 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:57.739038 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1422/2852] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:28:57.743225 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ffbf78-fa87-4288-99b9-96302b5b0acb admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:57.753685 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ffbf78-fa87-4288-99b9-96302b5b0acb admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:57.762858 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de94a2-b416-47d7-80ea-18ea333a420c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:57.763772 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de94a2-b416-47d7-80ea-18ea333a420c admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:57.764054 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de94a2-b416-47d7-80ea-18ea333a420c admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:57.764325 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de94a2-b416-47d7-80ea-18ea333a420c admin admin] PATH_INFO: `/v3/projects/9fcff3adb52a41e18b05a6408b87b9ac/users/a47f928e65d34451b93e1701430f2431/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:57.779286 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de94a2-b416-47d7-80ea-18ea333a420c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fcff3adb52a41e18b05a6408b87b9ac, user_id=a47f928e65d34451b93e1701430f2431, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:57.781956 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de94a2-b416-47d7-80ea-18ea333a420c admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:57.794850 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1431/2853] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:28:57 2022] PUT /identity/v3/projects/9fcff3adb52a41e18b05a6408b87b9ac/users/a47f928e65d34451b93e1701430f2431/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:57.806322 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc151b-dde0-4289-97f2-08cad86ae254 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:57.806856 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc151b-dde0-4289-97f2-08cad86ae254 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:57.806961 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc151b-dde0-4289-97f2-08cad86ae254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:57.843134 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-a6dc151b-dde0-4289-97f2-08cad86ae254 None None] MFA Rules not processed for user `a47f928e65d34451b93e1701430f2431`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:57.844402 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a6dc151b-dde0-4289-97f2-08cad86ae254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:57.915301 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1423/2854] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:57 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:57.918007 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de94a2-b416-47d7-80ea-18ea333a420c admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:57.920930 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-c7de94a2-b416-47d7-80ea-18ea333a420c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:57.927071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc151b-dde0-4289-97f2-08cad86ae254 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:57.927949 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-a6dc151b-dde0-4289-97f2-08cad86ae254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:57.976089 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:57.985557 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de94a2-b416-47d7-80ea-18ea333a420c admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:57.989466 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc151b-dde0-4289-97f2-08cad86ae254 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:57.990315 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30532aa3-5d79-4deb-93d4-f41bbc7b5ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:57.991319 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30532aa3-5d79-4deb-93d4-f41bbc7b5ede admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:57.991635 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30532aa3-5d79-4deb-93d4-f41bbc7b5ede admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:57.991911 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30532aa3-5d79-4deb-93d4-f41bbc7b5ede admin admin] PATH_INFO: `/v3/users/60783965056349f0978bfd6dc9f55435` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:57.993097 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30532aa3-5d79-4deb-93d4-f41bbc7b5ede admin admin] RBAC: Authorizing `identity:delete_user(user_id=60783965056349f0978bfd6dc9f55435)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:57.993318 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3142e316-b227-4735-b7cb-93fec86f6015 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2ce9e6497fdb4068b23aa3f7b2c19536', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:57.994087 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142e316-b227-4735-b7cb-93fec86f6015 service glance] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:57.994374 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142e316-b227-4735-b7cb-93fec86f6015 service glance] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:57.994649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142e316-b227-4735-b7cb-93fec86f6015 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:57.995022 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30532aa3-5d79-4deb-93d4-f41bbc7b5ede admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:58.000326 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:58.004167 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3142e316-b227-4735-b7cb-93fec86f6015 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:58.005146 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3142e316-b227-4735-b7cb-93fec86f6015 service glance] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:58.016593 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1424/2855] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:57 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:28:58.052133 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-30532aa3-5d79-4deb-93d4-f41bbc7b5ede admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60783965056349f0978bfd6dc9f55435'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:58.052876 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-30532aa3-5d79-4deb-93d4-f41bbc7b5ede admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60783965056349f0978bfd6dc9f55435'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:58.065880 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-30532aa3-5d79-4deb-93d4-f41bbc7b5ede admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60783965056349f0978bfd6dc9f55435'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:58.077913 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-30532aa3-5d79-4deb-93d4-f41bbc7b5ede admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60783965056349f0978bfd6dc9f55435'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:58.078608 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-30532aa3-5d79-4deb-93d4-f41bbc7b5ede admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60783965056349f0978bfd6dc9f55435'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:58.093934 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1432/2856] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:28:57 2022] DELETE /identity/v3/users/60783965056349f0978bfd6dc9f55435 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:58.098425 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c04f27-aff6-49ec-b46e-00b8189feab6 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:58.098712 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c04f27-aff6-49ec-b46e-00b8189feab6 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:58.098984 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c04f27-aff6-49ec-b46e-00b8189feab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:58.116499 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-53c04f27-aff6-49ec-b46e-00b8189feab6 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:58.117696 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-53c04f27-aff6-49ec-b46e-00b8189feab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:58.183118 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1425/2857] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:28:58.193305 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30532aa3-5d79-4deb-93d4-f41bbc7b5ede admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:58.194726 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-30532aa3-5d79-4deb-93d4-f41bbc7b5ede admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:58.255733 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30532aa3-5d79-4deb-93d4-f41bbc7b5ede admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:58.260616 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e582f-7e25-4f04-a273-7885ff58ee1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:58.261294 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e582f-7e25-4f04-a273-7885ff58ee1c service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:58.262253 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e582f-7e25-4f04-a273-7885ff58ee1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:58.262542 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e582f-7e25-4f04-a273-7885ff58ee1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:58.272293 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052e582f-7e25-4f04-a273-7885ff58ee1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:58.273751 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052e582f-7e25-4f04-a273-7885ff58ee1c service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:58.283485 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1433/2858] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:28:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 23:28:58.568335 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c04f27-aff6-49ec-b46e-00b8189feab6 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:58.569846 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-53c04f27-aff6-49ec-b46e-00b8189feab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:58.582564 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c04f27-aff6-49ec-b46e-00b8189feab6 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:58.587247 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb1624b-4a89-463b-b05b-436ef627db45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:58.588141 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb1624b-4a89-463b-b05b-436ef627db45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:58.588422 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb1624b-4a89-463b-b05b-436ef627db45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:58.588701 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb1624b-4a89-463b-b05b-436ef627db45 admin admin] PATH_INFO: `/v3/projects/958e8e2a460d433c8276f37bd0aa2fe9` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:58.589956 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb1624b-4a89-463b-b05b-436ef627db45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=958e8e2a460d433c8276f37bd0aa2fe9)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:58.591999 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb1624b-4a89-463b-b05b-436ef627db45 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:58.641615 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7eb1624b-4a89-463b-b05b-436ef627db45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '958e8e2a460d433c8276f37bd0aa2fe9', 'request_id': 'req-7eb1624b-4a89-463b-b05b-436ef627db45'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:58.647885 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7eb1624b-4a89-463b-b05b-436ef627db45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '958e8e2a460d433c8276f37bd0aa2fe9', 'request_id': 'req-7eb1624b-4a89-463b-b05b-436ef627db45'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:58.656991 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7eb1624b-4a89-463b-b05b-436ef627db45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '958e8e2a460d433c8276f37bd0aa2fe9', 'request_id': 'req-7eb1624b-4a89-463b-b05b-436ef627db45'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:58.667467 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7eb1624b-4a89-463b-b05b-436ef627db45 admin admin] The token cache is being invalidate because project 958e8e2a460d433c8276f37bd0aa2fe9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:28:58.668637 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7eb1624b-4a89-463b-b05b-436ef627db45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:58.669216 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-7eb1624b-4a89-463b-b05b-436ef627db45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:58.675422 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1426/2859] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:28:58 2022] DELETE /identity/v3/projects/958e8e2a460d433c8276f37bd0aa2fe9 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:58.675744 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e582f-7e25-4f04-a273-7885ff58ee1c service neutron] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:58.686070 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e582f-7e25-4f04-a273-7885ff58ee1c service neutron] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:58.747428 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8966d4b-0931-4dac-b7a2-17f4060e9475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:58.748341 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8966d4b-0931-4dac-b7a2-17f4060e9475 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:58.749276 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8966d4b-0931-4dac-b7a2-17f4060e9475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:58.749276 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8966d4b-0931-4dac-b7a2-17f4060e9475 admin admin] PATH_INFO: `/v3/users/f35d753aedba4fae886287715008433c` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:58.750644 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8966d4b-0931-4dac-b7a2-17f4060e9475 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f35d753aedba4fae886287715008433c)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:58.752707 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8966d4b-0931-4dac-b7a2-17f4060e9475 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:58.787885 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:58.809370 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-b8966d4b-0931-4dac-b7a2-17f4060e9475 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f35d753aedba4fae886287715008433c'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:58.810129 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-b8966d4b-0931-4dac-b7a2-17f4060e9475 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f35d753aedba4fae886287715008433c'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:58.831402 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-b8966d4b-0931-4dac-b7a2-17f4060e9475 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f35d753aedba4fae886287715008433c'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:58.838802 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-b8966d4b-0931-4dac-b7a2-17f4060e9475 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f35d753aedba4fae886287715008433c'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:58.840200 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-b8966d4b-0931-4dac-b7a2-17f4060e9475 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f35d753aedba4fae886287715008433c'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:58.854236 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1434/2860] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:28:58 2022] DELETE /identity/v3/users/f35d753aedba4fae886287715008433c => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:59.155602 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb1624b-4a89-463b-b05b-436ef627db45 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:59.157115 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-7eb1624b-4a89-463b-b05b-436ef627db45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:59.170818 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb1624b-4a89-463b-b05b-436ef627db45 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:59.231876 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81da51-f662-4a43-91c6-85a5f7425c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:59.232803 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81da51-f662-4a43-91c6-85a5f7425c22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:59.233090 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81da51-f662-4a43-91c6-85a5f7425c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:59.233498 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81da51-f662-4a43-91c6-85a5f7425c22 admin admin] PATH_INFO: `/v3/projects/bcff634b060a4b0e92725eeac37a95ea` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:59.234954 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d81da51-f662-4a43-91c6-85a5f7425c22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcff634b060a4b0e92725eeac37a95ea)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:59.237313 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d81da51-f662-4a43-91c6-85a5f7425c22 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:59.249176 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:28:59.288699 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1d81da51-f662-4a43-91c6-85a5f7425c22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcff634b060a4b0e92725eeac37a95ea', 'request_id': 'req-1d81da51-f662-4a43-91c6-85a5f7425c22'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:59.295014 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1d81da51-f662-4a43-91c6-85a5f7425c22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcff634b060a4b0e92725eeac37a95ea', 'request_id': 'req-1d81da51-f662-4a43-91c6-85a5f7425c22'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:59.302218 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1d81da51-f662-4a43-91c6-85a5f7425c22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcff634b060a4b0e92725eeac37a95ea', 'request_id': 'req-1d81da51-f662-4a43-91c6-85a5f7425c22'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:59.312403 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1d81da51-f662-4a43-91c6-85a5f7425c22 admin admin] The token cache is being invalidate because project bcff634b060a4b0e92725eeac37a95ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:28:59.312685 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1d81da51-f662-4a43-91c6-85a5f7425c22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:59.313300 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-1d81da51-f662-4a43-91c6-85a5f7425c22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:59.314843 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1427/2861] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:28:59 2022] DELETE /identity/v3/projects/bcff634b060a4b0e92725eeac37a95ea => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:59.319096 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8966d4b-0931-4dac-b7a2-17f4060e9475 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:59.320553 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-b8966d4b-0931-4dac-b7a2-17f4060e9475 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:59.334215 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8966d4b-0931-4dac-b7a2-17f4060e9475 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:59.391485 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5827e9d-dac8-4b62-97e8-b4958386586f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:59.392391 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5827e9d-dac8-4b62-97e8-b4958386586f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:59.392670 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5827e9d-dac8-4b62-97e8-b4958386586f admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:59.392942 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5827e9d-dac8-4b62-97e8-b4958386586f admin admin] PATH_INFO: `/v3/users/2408c2a7c8f947fb9686a5b5368fd9c6` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:59.394112 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5827e9d-dac8-4b62-97e8-b4958386586f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2408c2a7c8f947fb9686a5b5368fd9c6)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:59.396424 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5827e9d-dac8-4b62-97e8-b4958386586f admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:59.444989 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-d5827e9d-dac8-4b62-97e8-b4958386586f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2408c2a7c8f947fb9686a5b5368fd9c6'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:59.445671 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-d5827e9d-dac8-4b62-97e8-b4958386586f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2408c2a7c8f947fb9686a5b5368fd9c6'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:59.458536 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-d5827e9d-dac8-4b62-97e8-b4958386586f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2408c2a7c8f947fb9686a5b5368fd9c6'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:59.465957 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-d5827e9d-dac8-4b62-97e8-b4958386586f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2408c2a7c8f947fb9686a5b5368fd9c6'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:59.466608 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-d5827e9d-dac8-4b62-97e8-b4958386586f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2408c2a7c8f947fb9686a5b5368fd9c6'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:59.481000 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1435/2862] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:28:59 2022] DELETE /identity/v3/users/2408c2a7c8f947fb9686a5b5368fd9c6 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:28:59.769118 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81da51-f662-4a43-91c6-85a5f7425c22 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:28:59.770694 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-1d81da51-f662-4a43-91c6-85a5f7425c22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:59.785365 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81da51-f662-4a43-91c6-85a5f7425c22 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:28:59.843999 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f71d4f-cd27-4bf6-ac16-19793f2fa1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:28:59.844872 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f71d4f-cd27-4bf6-ac16-19793f2fa1b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:59.845136 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f71d4f-cd27-4bf6-ac16-19793f2fa1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:59.845408 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f71d4f-cd27-4bf6-ac16-19793f2fa1b5 admin admin] PATH_INFO: `/v3/projects/47d9afb8dc9344ac937e2af8cdbac461` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:59.846599 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f71d4f-cd27-4bf6-ac16-19793f2fa1b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47d9afb8dc9344ac937e2af8cdbac461)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:28:59.848576 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f71d4f-cd27-4bf6-ac16-19793f2fa1b5 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:28:59.898313 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c8f71d4f-cd27-4bf6-ac16-19793f2fa1b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47d9afb8dc9344ac937e2af8cdbac461', 'request_id': 'req-c8f71d4f-cd27-4bf6-ac16-19793f2fa1b5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:59.905157 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c8f71d4f-cd27-4bf6-ac16-19793f2fa1b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47d9afb8dc9344ac937e2af8cdbac461', 'request_id': 'req-c8f71d4f-cd27-4bf6-ac16-19793f2fa1b5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:59.913851 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c8f71d4f-cd27-4bf6-ac16-19793f2fa1b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47d9afb8dc9344ac937e2af8cdbac461', 'request_id': 'req-c8f71d4f-cd27-4bf6-ac16-19793f2fa1b5'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:59.926349 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c8f71d4f-cd27-4bf6-ac16-19793f2fa1b5 admin admin] The token cache is being invalidate because project 47d9afb8dc9344ac937e2af8cdbac461 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:28:59.926634 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c8f71d4f-cd27-4bf6-ac16-19793f2fa1b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:59.927286 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-c8f71d4f-cd27-4bf6-ac16-19793f2fa1b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:28:59.929539 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1428/2863] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:28:59 2022] DELETE /identity/v3/projects/47d9afb8dc9344ac937e2af8cdbac461 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:28:59.935471 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8115c4-9ab1-407d-86ec-42129b593ea6 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:28:59.936171 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8115c4-9ab1-407d-86ec-42129b593ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:28:59.936298 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8115c4-9ab1-407d-86ec-42129b593ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:28:59.957923 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-3b8115c4-9ab1-407d-86ec-42129b593ea6 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:28:59.959163 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-3b8115c4-9ab1-407d-86ec-42129b593ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:28:59.980748 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:29:00.029656 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1436/2864] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:28:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:29:00.037835 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495d313-df6d-43c0-a379-aa8bd61cdef2 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:29:00.038131 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495d313-df6d-43c0-a379-aa8bd61cdef2 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:29:00.038416 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495d313-df6d-43c0-a379-aa8bd61cdef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:29:00.057638 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-4495d313-df6d-43c0-a379-aa8bd61cdef2 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:29:00.059527 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-4495d313-df6d-43c0-a379-aa8bd61cdef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:29:00.067517 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1429/2865] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:29:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:29:00.071743 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8115c4-9ab1-407d-86ec-42129b593ea6 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:29:00.081958 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8115c4-9ab1-407d-86ec-42129b593ea6 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:29:00.086591 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f56fc-90c6-423e-9ba3-db9ad7edf16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:29:00.087252 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f56fc-90c6-423e-9ba3-db9ad7edf16f admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:29:00.087560 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f56fc-90c6-423e-9ba3-db9ad7edf16f admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:29:00.087831 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f56fc-90c6-423e-9ba3-db9ad7edf16f admin admin] PATH_INFO: `/v3/domains` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:29:00.088554 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113f56fc-90c6-423e-9ba3-db9ad7edf16f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80285) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:29:00.088857 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113f56fc-90c6-423e-9ba3-db9ad7edf16f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:29:00.090495 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113f56fc-90c6-423e-9ba3-db9ad7edf16f admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:29:00.103598 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1437/2866] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:29:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:29:00.107828 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495d313-df6d-43c0-a379-aa8bd61cdef2 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:29:00.117411 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495d313-df6d-43c0-a379-aa8bd61cdef2 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:29:00.122857 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5e0fd-3305-462d-8998-f4d6f7a2d96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:29:00.123615 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5e0fd-3305-462d-8998-f4d6f7a2d96f admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:29:00.123894 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5e0fd-3305-462d-8998-f4d6f7a2d96f admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:29:00.124167 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5e0fd-3305-462d-8998-f4d6f7a2d96f admin admin] PATH_INFO: `/v3/projects` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:29:00.124874 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f5e0fd-3305-462d-8998-f4d6f7a2d96f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:29:00.126810 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f5e0fd-3305-462d-8998-f4d6f7a2d96f admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:29:00.146302 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:29:00.153954 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1430/2867] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:29:00 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:29:00.158252 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f56fc-90c6-423e-9ba3-db9ad7edf16f admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:29:00.170322 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f56fc-90c6-423e-9ba3-db9ad7edf16f admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:29:00.225695 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a62de-f72f-4049-aa07-914d41a91883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:29:00.226460 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a62de-f72f-4049-aa07-914d41a91883 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:29:00.226751 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a62de-f72f-4049-aa07-914d41a91883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:29:00.227063 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a62de-f72f-4049-aa07-914d41a91883 admin admin] PATH_INFO: `/v3/users` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:29:00.227781 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83a62de-f72f-4049-aa07-914d41a91883 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:29:00.229725 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83a62de-f72f-4049-aa07-914d41a91883 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:29:00.252056 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1438/2868] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:29:00 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:29:00.256215 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5e0fd-3305-462d-8998-f4d6f7a2d96f admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:29:00.266780 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5e0fd-3305-462d-8998-f4d6f7a2d96f admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:29:00.272347 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfb0115-c499-4331-a2d7-aef9f3a7d996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:29:00.273165 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb0115-c499-4331-a2d7-aef9f3a7d996 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:29:00.273509 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb0115-c499-4331-a2d7-aef9f3a7d996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:29:00.273788 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb0115-c499-4331-a2d7-aef9f3a7d996 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:29:00.274522 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfb0115-c499-4331-a2d7-aef9f3a7d996 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:29:00.276206 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfb0115-c499-4331-a2d7-aef9f3a7d996 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:29:00.287794 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1431/2869] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:29:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:29:00.291974 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a62de-f72f-4049-aa07-914d41a91883 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:29:00.302772 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a62de-f72f-4049-aa07-914d41a91883 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:29:00.307292 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd9add-9b50-4c82-939d-25783f1ce9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:29:00.307978 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd9add-9b50-4c82-939d-25783f1ce9da admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:29:00.308258 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd9add-9b50-4c82-939d-25783f1ce9da admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:29:00.308529 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd9add-9b50-4c82-939d-25783f1ce9da admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:29:00.309259 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dd9add-9b50-4c82-939d-25783f1ce9da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:29:00.310968 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dd9add-9b50-4c82-939d-25783f1ce9da admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:29:00.323607 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1439/2870] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:29:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:29:00.327794 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfb0115-c499-4331-a2d7-aef9f3a7d996 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:29:00.337822 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfb0115-c499-4331-a2d7-aef9f3a7d996 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:29:00.343060 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8756dea-df6c-4c56-a884-16e1099425b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:29:00.343974 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8756dea-df6c-4c56-a884-16e1099425b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:29:00.344258 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8756dea-df6c-4c56-a884-16e1099425b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:29:00.344536 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8756dea-df6c-4c56-a884-16e1099425b6 admin admin] PATH_INFO: `/v3/projects/919e242055ec419a921ba07ce05e1b6a/users/8b7ec4ead25745d49593a6ddcfe35b11/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:29:00.360767 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8756dea-df6c-4c56-a884-16e1099425b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=919e242055ec419a921ba07ce05e1b6a, user_id=8b7ec4ead25745d49593a6ddcfe35b11, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:29:00.363470 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8756dea-df6c-4c56-a884-16e1099425b6 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:29:00.376475 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1432/2871] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:29:00 2022] PUT /identity/v3/projects/919e242055ec419a921ba07ce05e1b6a/users/8b7ec4ead25745d49593a6ddcfe35b11/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:29:00.387821 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211da4d-4b77-4f95-bb6e-91418d527ed1 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:29:00.388154 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211da4d-4b77-4f95-bb6e-91418d527ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:29:00.388457 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211da4d-4b77-4f95-bb6e-91418d527ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:29:00.416049 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-c211da4d-4b77-4f95-bb6e-91418d527ed1 None None] MFA Rules not processed for user `8b7ec4ead25745d49593a6ddcfe35b11`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:29:00.417411 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-c211da4d-4b77-4f95-bb6e-91418d527ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:29:00.483665 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1440/2872] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:29:00 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:29:00.494681 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8756dea-df6c-4c56-a884-16e1099425b6 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:29:00.496416 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f8756dea-df6c-4c56-a884-16e1099425b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:29:00.556632 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8756dea-df6c-4c56-a884-16e1099425b6 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:29:00.560489 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26852834-b5b9-4e71-b410-69518b208f7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:29:00.561137 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26852834-b5b9-4e71-b410-69518b208f7f service cinder] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:29:00.561420 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26852834-b5b9-4e71-b410-69518b208f7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:29:00.561824 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26852834-b5b9-4e71-b410-69518b208f7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:29:00.573511 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26852834-b5b9-4e71-b410-69518b208f7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:29:00.573511 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26852834-b5b9-4e71-b410-69518b208f7f service cinder] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:29:00.582820 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1433/2873] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:29:00 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:29:10.479898 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=88,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:29:10.578604 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:29:23.932541 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211da4d-4b77-4f95-bb6e-91418d527ed1 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:29:23.934117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-c211da4d-4b77-4f95-bb6e-91418d527ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:29:23.995507 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211da4d-4b77-4f95-bb6e-91418d527ed1 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:29:23.999318 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9854c7-73f5-4052-8d72-e1999db8c9ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:29:24.000014 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9854c7-73f5-4052-8d72-e1999db8c9ab service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:29:24.000296 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9854c7-73f5-4052-8d72-e1999db8c9ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:29:24.000573 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9854c7-73f5-4052-8d72-e1999db8c9ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:29:24.002521 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-ed9854c7-73f5-4052-8d72-e1999db8c9ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:29:24.072211 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9854c7-73f5-4052-8d72-e1999db8c9ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:29:24.073674 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9854c7-73f5-4052-8d72-e1999db8c9ab service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:29:24.083802 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1441/2874] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:29:23 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:29:34.079402 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=50 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:29:56.536981 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26852834-b5b9-4e71-b410-69518b208f7f service cinder] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:29:56.538514 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-26852834-b5b9-4e71-b410-69518b208f7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:29:56.728188 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26852834-b5b9-4e71-b410-69518b208f7f service cinder] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:29:56.734278 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b6778-e54f-4fb9-8953-d6cc8eea01ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:29:56.735136 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b6778-e54f-4fb9-8953-d6cc8eea01ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:29:56.735415 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b6778-e54f-4fb9-8953-d6cc8eea01ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:29:56.735728 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b6778-e54f-4fb9-8953-d6cc8eea01ee admin admin] PATH_INFO: `/v3/users/a47f928e65d34451b93e1701430f2431` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:29:56.736951 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2b6778-e54f-4fb9-8953-d6cc8eea01ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=a47f928e65d34451b93e1701430f2431)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:29:56.738838 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2b6778-e54f-4fb9-8953-d6cc8eea01ee admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:29:56.784195 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0a2b6778-e54f-4fb9-8953-d6cc8eea01ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a47f928e65d34451b93e1701430f2431'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:29:56.784770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0a2b6778-e54f-4fb9-8953-d6cc8eea01ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a47f928e65d34451b93e1701430f2431'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:29:56.797661 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0a2b6778-e54f-4fb9-8953-d6cc8eea01ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a47f928e65d34451b93e1701430f2431'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:29:56.804607 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0a2b6778-e54f-4fb9-8953-d6cc8eea01ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a47f928e65d34451b93e1701430f2431'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:29:56.812589 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-0a2b6778-e54f-4fb9-8953-d6cc8eea01ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a47f928e65d34451b93e1701430f2431'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:29:56.818550 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1434/2875] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:29:56 2022] DELETE /identity/v3/users/a47f928e65d34451b93e1701430f2431 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:29:56.823092 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6480d1c5-6190-49f2-9059-07ad34e93eb9 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:29:56.823634 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6480d1c5-6190-49f2-9059-07ad34e93eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:29:56.823749 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6480d1c5-6190-49f2-9059-07ad34e93eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:29:56.840256 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-6480d1c5-6190-49f2-9059-07ad34e93eb9 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:29:56.841458 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-6480d1c5-6190-49f2-9059-07ad34e93eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:29:56.899282 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1442/2876] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:29:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:29:56.909670 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b6778-e54f-4fb9-8953-d6cc8eea01ee admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:29:56.911061 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-0a2b6778-e54f-4fb9-8953-d6cc8eea01ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:29:56.968649 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b6778-e54f-4fb9-8953-d6cc8eea01ee admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:29:56.972306 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c330e0-e12e-4b4e-896d-51e5fe2b29f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:29:56.973154 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c330e0-e12e-4b4e-896d-51e5fe2b29f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:29:56.973444 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c330e0-e12e-4b4e-896d-51e5fe2b29f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:29:56.973832 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c330e0-e12e-4b4e-896d-51e5fe2b29f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:29:56.983926 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c330e0-e12e-4b4e-896d-51e5fe2b29f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:29:56.985328 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c330e0-e12e-4b4e-896d-51e5fe2b29f9 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:29:56.994783 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1435/2877] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:29:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:29:57.269117 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6480d1c5-6190-49f2-9059-07ad34e93eb9 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:29:57.270825 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-6480d1c5-6190-49f2-9059-07ad34e93eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:29:57.284876 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6480d1c5-6190-49f2-9059-07ad34e93eb9 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:29:57.289604 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0dd5d5-f660-4daa-b143-9345a1ec2b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:29:57.290471 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0dd5d5-f660-4daa-b143-9345a1ec2b55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:29:57.290767 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0dd5d5-f660-4daa-b143-9345a1ec2b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:29:57.291093 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0dd5d5-f660-4daa-b143-9345a1ec2b55 admin admin] PATH_INFO: `/v3/projects/9fcff3adb52a41e18b05a6408b87b9ac` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:29:57.292561 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0dd5d5-f660-4daa-b143-9345a1ec2b55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fcff3adb52a41e18b05a6408b87b9ac)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:29:57.294907 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0dd5d5-f660-4daa-b143-9345a1ec2b55 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:29:57.348617 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-6f0dd5d5-f660-4daa-b143-9345a1ec2b55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fcff3adb52a41e18b05a6408b87b9ac', 'request_id': 'req-6f0dd5d5-f660-4daa-b143-9345a1ec2b55'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:29:57.356511 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-6f0dd5d5-f660-4daa-b143-9345a1ec2b55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fcff3adb52a41e18b05a6408b87b9ac', 'request_id': 'req-6f0dd5d5-f660-4daa-b143-9345a1ec2b55'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:29:57.369241 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-6f0dd5d5-f660-4daa-b143-9345a1ec2b55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fcff3adb52a41e18b05a6408b87b9ac', 'request_id': 'req-6f0dd5d5-f660-4daa-b143-9345a1ec2b55'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:29:57.381872 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-6f0dd5d5-f660-4daa-b143-9345a1ec2b55 admin admin] The token cache is being invalidate because project 9fcff3adb52a41e18b05a6408b87b9ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:29:57.382045 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-6f0dd5d5-f660-4daa-b143-9345a1ec2b55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:29:57.382761 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-6f0dd5d5-f660-4daa-b143-9345a1ec2b55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:29:57.384383 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1443/2878] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:29:57 2022] DELETE /identity/v3/projects/9fcff3adb52a41e18b05a6408b87b9ac => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:30:00.711918 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c330e0-e12e-4b4e-896d-51e5fe2b29f9 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:30:00.713438 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b1c330e0-e12e-4b4e-896d-51e5fe2b29f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:30:00.726683 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c330e0-e12e-4b4e-896d-51e5fe2b29f9 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:30:00.744959 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:30:00.784452 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d48c3-d064-403b-9642-cbd463e55c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:30:00.785310 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d48c3-d064-403b-9642-cbd463e55c16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:30:00.785592 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d48c3-d064-403b-9642-cbd463e55c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:30:00.785866 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d48c3-d064-403b-9642-cbd463e55c16 admin admin] PATH_INFO: `/v3/users/8b7ec4ead25745d49593a6ddcfe35b11` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:30:00.787060 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d48c3-d064-403b-9642-cbd463e55c16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b7ec4ead25745d49593a6ddcfe35b11)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:30:00.788994 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d48c3-d064-403b-9642-cbd463e55c16 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:30:00.834461 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3a5d48c3-d064-403b-9642-cbd463e55c16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b7ec4ead25745d49593a6ddcfe35b11'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:30:00.835112 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3a5d48c3-d064-403b-9642-cbd463e55c16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b7ec4ead25745d49593a6ddcfe35b11'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:30:00.847055 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3a5d48c3-d064-403b-9642-cbd463e55c16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b7ec4ead25745d49593a6ddcfe35b11'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:30:00.853877 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3a5d48c3-d064-403b-9642-cbd463e55c16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b7ec4ead25745d49593a6ddcfe35b11'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:30:00.861354 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-3a5d48c3-d064-403b-9642-cbd463e55c16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b7ec4ead25745d49593a6ddcfe35b11'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:30:00.867320 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1436/2879] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:30:00 2022] DELETE /identity/v3/users/8b7ec4ead25745d49593a6ddcfe35b11 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:30:00.871980 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98aab3-11b5-4fd0-bc7b-92def4363843 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:30:00.872479 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98aab3-11b5-4fd0-bc7b-92def4363843 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:30:00.872619 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e98aab3-11b5-4fd0-bc7b-92def4363843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:30:00.890102 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-0e98aab3-11b5-4fd0-bc7b-92def4363843 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:30:00.891316 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-0e98aab3-11b5-4fd0-bc7b-92def4363843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:30:00.949590 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1444/2880] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:30:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:30:00.959715 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d48c3-d064-403b-9642-cbd463e55c16 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:30:00.961239 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-3a5d48c3-d064-403b-9642-cbd463e55c16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:30:01.018680 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d48c3-d064-403b-9642-cbd463e55c16 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:30:01.022473 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178f92b-dd0d-4ac4-bd52-5399b951aaa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:30:01.023121 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178f92b-dd0d-4ac4-bd52-5399b951aaa0 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:30:01.023401 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178f92b-dd0d-4ac4-bd52-5399b951aaa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:30:01.023711 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178f92b-dd0d-4ac4-bd52-5399b951aaa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:30:01.033637 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c178f92b-dd0d-4ac4-bd52-5399b951aaa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:30:01.035065 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c178f92b-dd0d-4ac4-bd52-5399b951aaa0 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:30:01.045460 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1437/2881] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:30:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:30:01.316441 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e98aab3-11b5-4fd0-bc7b-92def4363843 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:30:01.317967 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-0e98aab3-11b5-4fd0-bc7b-92def4363843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:30:01.332902 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e98aab3-11b5-4fd0-bc7b-92def4363843 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:30:01.338371 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6be260-a2f8-40ab-8e76-12aed7635119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:30:01.339245 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6be260-a2f8-40ab-8e76-12aed7635119 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:30:01.339554 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6be260-a2f8-40ab-8e76-12aed7635119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:30:01.339934 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6be260-a2f8-40ab-8e76-12aed7635119 admin admin] PATH_INFO: `/v3/projects/919e242055ec419a921ba07ce05e1b6a` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:30:01.341057 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6be260-a2f8-40ab-8e76-12aed7635119 admin admin] RBAC: Authorizing `identity:delete_project(project_id=919e242055ec419a921ba07ce05e1b6a)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:30:01.342982 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6be260-a2f8-40ab-8e76-12aed7635119 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:30:01.365482 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:30:01.391697 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-8b6be260-a2f8-40ab-8e76-12aed7635119 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '919e242055ec419a921ba07ce05e1b6a', 'request_id': 'req-8b6be260-a2f8-40ab-8e76-12aed7635119'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:30:01.398145 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-8b6be260-a2f8-40ab-8e76-12aed7635119 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '919e242055ec419a921ba07ce05e1b6a', 'request_id': 'req-8b6be260-a2f8-40ab-8e76-12aed7635119'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:30:01.406084 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-8b6be260-a2f8-40ab-8e76-12aed7635119 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '919e242055ec419a921ba07ce05e1b6a', 'request_id': 'req-8b6be260-a2f8-40ab-8e76-12aed7635119'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:30:01.416731 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-8b6be260-a2f8-40ab-8e76-12aed7635119 admin admin] The token cache is being invalidate because project 919e242055ec419a921ba07ce05e1b6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:30:01.417018 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-8b6be260-a2f8-40ab-8e76-12aed7635119 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:30:01.417657 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-8b6be260-a2f8-40ab-8e76-12aed7635119 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:30:01.419455 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1445/2882] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:30:01 2022] DELETE /identity/v3/projects/919e242055ec419a921ba07ce05e1b6a => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:30:01.424302 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04556045-5435-46ed-9a5e-775fe1954bba None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:30:01.424627 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04556045-5435-46ed-9a5e-775fe1954bba None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:30:01.424931 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04556045-5435-46ed-9a5e-775fe1954bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:30:01.442881 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-04556045-5435-46ed-9a5e-775fe1954bba None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:30:01.444130 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-04556045-5435-46ed-9a5e-775fe1954bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:30:01.484354 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:30:01.506058 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1438/2883] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:30:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:30:01.515047 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca9864-24d9-4052-9b1c-59246ff4f1dd None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:30:01.515330 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca9864-24d9-4052-9b1c-59246ff4f1dd None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:30:01.515640 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca9864-24d9-4052-9b1c-59246ff4f1dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:30:01.535243 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-54ca9864-24d9-4052-9b1c-59246ff4f1dd None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:30:01.536510 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-54ca9864-24d9-4052-9b1c-59246ff4f1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:30:01.544821 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1446/2884] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:30:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 23:30:01.549495 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04556045-5435-46ed-9a5e-775fe1954bba None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:30:01.559165 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04556045-5435-46ed-9a5e-775fe1954bba None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:30:01.564401 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c926ac0-71e8-4efd-96bc-28be981906d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:30:01.565037 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c926ac0-71e8-4efd-96bc-28be981906d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:30:01.565314 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c926ac0-71e8-4efd-96bc-28be981906d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:30:01.565587 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c926ac0-71e8-4efd-96bc-28be981906d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:30:01.566298 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c926ac0-71e8-4efd-96bc-28be981906d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80284) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 23:30:01.566600 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c926ac0-71e8-4efd-96bc-28be981906d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:30:01.568275 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c926ac0-71e8-4efd-96bc-28be981906d6 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:30:01.580148 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1439/2885] 10.222.0.60 () {58 vars in 1152 bytes} [Wed Jul 13 23:30:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 23:30:01.584299 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca9864-24d9-4052-9b1c-59246ff4f1dd None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:30:01.593754 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca9864-24d9-4052-9b1c-59246ff4f1dd None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:30:01.598431 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858110e-3afd-41b5-8fd9-fce761c498b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:30:01.599080 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858110e-3afd-41b5-8fd9-fce761c498b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:30:01.599363 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858110e-3afd-41b5-8fd9-fce761c498b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:30:01.599679 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858110e-3afd-41b5-8fd9-fce761c498b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:30:01.600378 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1858110e-3afd-41b5-8fd9-fce761c498b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:30:01.602286 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1858110e-3afd-41b5-8fd9-fce761c498b6 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:30:01.628374 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1447/2886] 10.222.0.60 () {60 vars in 1152 bytes} [Wed Jul 13 23:30:01 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:30:01.632521 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c926ac0-71e8-4efd-96bc-28be981906d6 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:30:01.643408 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c926ac0-71e8-4efd-96bc-28be981906d6 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:30:01.699071 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a330a106-85ae-455b-b396-a3e947892476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:30:01.699766 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a330a106-85ae-455b-b396-a3e947892476 admin admin] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:30:01.700046 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a330a106-85ae-455b-b396-a3e947892476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:30:01.700320 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a330a106-85ae-455b-b396-a3e947892476 admin admin] PATH_INFO: `/v3/users` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:30:01.701007 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a330a106-85ae-455b-b396-a3e947892476 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:30:01.702890 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a330a106-85ae-455b-b396-a3e947892476 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:30:01.721928 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1440/2887] 10.222.0.60 () {60 vars in 1143 bytes} [Wed Jul 13 23:30:01 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 23:30:01.726038 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858110e-3afd-41b5-8fd9-fce761c498b6 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:30:01.736959 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858110e-3afd-41b5-8fd9-fce761c498b6 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:30:01.741523 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d09a87e-6b2a-42a7-b146-16c513b62a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:30:01.742172 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09a87e-6b2a-42a7-b146-16c513b62a65 admin admin] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:30:01.742454 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09a87e-6b2a-42a7-b146-16c513b62a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:30:01.742727 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d09a87e-6b2a-42a7-b146-16c513b62a65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:30:01.743443 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d09a87e-6b2a-42a7-b146-16c513b62a65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:30:01.745101 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d09a87e-6b2a-42a7-b146-16c513b62a65 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:30:01.756126 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1448/2888] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 23:30:01.759962 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a330a106-85ae-455b-b396-a3e947892476 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:30:01.769811 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a330a106-85ae-455b-b396-a3e947892476 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:30:01.774372 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f32b3-08a8-4f67-88a0-852f46e39eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:30:01.775010 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f32b3-08a8-4f67-88a0-852f46e39eb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:30:01.775289 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f32b3-08a8-4f67-88a0-852f46e39eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:30:01.775587 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f32b3-08a8-4f67-88a0-852f46e39eb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:30:01.776310 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28f32b3-08a8-4f67-88a0-852f46e39eb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:30:01.777879 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28f32b3-08a8-4f67-88a0-852f46e39eb4 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:30:01.787604 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1441/2889] 10.222.0.60 () {58 vars in 1121 bytes} [Wed Jul 13 23:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 23:30:01.791707 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d09a87e-6b2a-42a7-b146-16c513b62a65 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:30:01.802426 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d09a87e-6b2a-42a7-b146-16c513b62a65 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:30:01.807714 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6fe7e8-32fb-4858-bd09-197bbb500364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:30:01.808624 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6fe7e8-32fb-4858-bd09-197bbb500364 admin admin] REQUEST_METHOD: `PUT` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:30:01.808896 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6fe7e8-32fb-4858-bd09-197bbb500364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:30:01.809200 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6fe7e8-32fb-4858-bd09-197bbb500364 admin admin] PATH_INFO: `/v3/projects/18e41a3e7f2147329cfc9a9f7b0c1036/users/c366507b025248dabbfed7c1ff4e3cc9/roles/f005bdf39a0743238b3476f6b88b3684` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:30:01.825307 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6fe7e8-32fb-4858-bd09-197bbb500364 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18e41a3e7f2147329cfc9a9f7b0c1036, user_id=c366507b025248dabbfed7c1ff4e3cc9, role_id=f005bdf39a0743238b3476f6b88b3684)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:30:01.828187 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6fe7e8-32fb-4858-bd09-197bbb500364 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:30:01.840467 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1449/2890] 10.222.0.60 () {60 vars in 1482 bytes} [Wed Jul 13 23:30:01 2022] PUT /identity/v3/projects/18e41a3e7f2147329cfc9a9f7b0c1036/users/c366507b025248dabbfed7c1ff4e3cc9/roles/f005bdf39a0743238b3476f6b88b3684 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:30:01.851591 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e13271-914d-43d5-82ec-4f93c1a525fa None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:30:01.851884 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e13271-914d-43d5-82ec-4f93c1a525fa None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:30:01.852163 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e13271-914d-43d5-82ec-4f93c1a525fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:30:01.878581 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-e2e13271-914d-43d5-82ec-4f93c1a525fa None None] MFA Rules not processed for user `c366507b025248dabbfed7c1ff4e3cc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:30:01.879833 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e2e13271-914d-43d5-82ec-4f93c1a525fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:30:01.942460 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1442/2891] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:30:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:30:01.953877 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6fe7e8-32fb-4858-bd09-197bbb500364 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:30:01.955676 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-5b6fe7e8-32fb-4858-bd09-197bbb500364 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:30:02.011883 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6fe7e8-32fb-4858-bd09-197bbb500364 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:30:02.015497 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0dc9d0-06fb-4cb3-bd55-73c5a8b000a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54661bad914f455cb00d42ec692fde11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:30:02.016170 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0dc9d0-06fb-4cb3-bd55-73c5a8b000a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:30:02.016456 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0dc9d0-06fb-4cb3-bd55-73c5a8b000a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:30:02.016736 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0dc9d0-06fb-4cb3-bd55-73c5a8b000a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:30:02.025025 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0dc9d0-06fb-4cb3-bd55-73c5a8b000a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:30:02.026471 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0dc9d0-06fb-4cb3-bd55-73c5a8b000a0 service cinder] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:30:02.035347 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1450/2892] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:30:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:30:02.060275 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e13271-914d-43d5-82ec-4f93c1a525fa None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:30:02.104185 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:30:02.124965 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e13271-914d-43d5-82ec-4f93c1a525fa None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:30:02.129639 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a9856a-bb3b-4592-9a9c-634fdd528813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:30:02.130508 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9856a-bb3b-4592-9a9c-634fdd528813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:30:02.130790 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9856a-bb3b-4592-9a9c-634fdd528813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:30:02.131062 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a9856a-bb3b-4592-9a9c-634fdd528813 admin admin] PATH_INFO: `/v3/users/c366507b025248dabbfed7c1ff4e3cc9` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:30:02.132285 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a9856a-bb3b-4592-9a9c-634fdd528813 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c366507b025248dabbfed7c1ff4e3cc9)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:30:02.134202 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a9856a-bb3b-4592-9a9c-634fdd528813 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:30:02.180038 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e6a9856a-bb3b-4592-9a9c-634fdd528813 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c366507b025248dabbfed7c1ff4e3cc9'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:30:02.180717 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e6a9856a-bb3b-4592-9a9c-634fdd528813 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c366507b025248dabbfed7c1ff4e3cc9'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:30:02.192441 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e6a9856a-bb3b-4592-9a9c-634fdd528813 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c366507b025248dabbfed7c1ff4e3cc9'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:30:02.199431 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e6a9856a-bb3b-4592-9a9c-634fdd528813 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c366507b025248dabbfed7c1ff4e3cc9'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:30:02.206304 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-e6a9856a-bb3b-4592-9a9c-634fdd528813 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c366507b025248dabbfed7c1ff4e3cc9'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:30:02.212347 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1443/2893] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:30:02 2022] DELETE /identity/v3/users/c366507b025248dabbfed7c1ff4e3cc9 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:30:02.216882 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4015bd-c86b-42e1-bf36-f1f94bf4cb05 None None] REQUEST_METHOD: `POST` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:30:02.217219 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4015bd-c86b-42e1-bf36-f1f94bf4cb05 None None] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:30:02.217990 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4015bd-c86b-42e1-bf36-f1f94bf4cb05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:30:02.235575 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.auth.core [None req-ff4015bd-c86b-42e1-bf36-f1f94bf4cb05 None None] MFA Rules not processed for user `7f1cf8ea49bb4e7fb83c77a4a463d6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80285) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:30:02.236793 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-ff4015bd-c86b-42e1-bf36-f1f94bf4cb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:30:02.278114 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:30:02.298128 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1451/2894] 10.222.0.60 () {58 vars in 957 bytes} [Wed Jul 13 23:30:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:30:02.307770 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a9856a-bb3b-4592-9a9c-634fdd528813 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:30:02.309187 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e6a9856a-bb3b-4592-9a9c-634fdd528813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:30:02.367375 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a9856a-bb3b-4592-9a9c-634fdd528813 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:30:02.371194 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ada2c3-cbac-4f28-8630-095f989414f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:30:02.371869 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ada2c3-cbac-4f28-8630-095f989414f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:30:02.372153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ada2c3-cbac-4f28-8630-095f989414f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:30:02.372428 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ada2c3-cbac-4f28-8630-095f989414f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:30:02.381804 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ada2c3-cbac-4f28-8630-095f989414f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:30:02.383233 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ada2c3-cbac-4f28-8630-095f989414f0 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:30:02.392623 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1444/2895] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:30:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:30:02.658946 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4015bd-c86b-42e1-bf36-f1f94bf4cb05 None None] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:30:02.660506 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-ff4015bd-c86b-42e1-bf36-f1f94bf4cb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:30:02.676694 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4015bd-c86b-42e1-bf36-f1f94bf4cb05 None None] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:30:02.681530 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00666838-14f8-4a95-a740-010dcf56cc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:30:02.682590 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00666838-14f8-4a95-a740-010dcf56cc90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:30:02.682734 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00666838-14f8-4a95-a740-010dcf56cc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:30:02.682957 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00666838-14f8-4a95-a740-010dcf56cc90 admin admin] PATH_INFO: `/v3/projects/18e41a3e7f2147329cfc9a9f7b0c1036` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:30:02.684224 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00666838-14f8-4a95-a740-010dcf56cc90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18e41a3e7f2147329cfc9a9f7b0c1036)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:30:02.686089 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00666838-14f8-4a95-a740-010dcf56cc90 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:30:02.733601 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-00666838-14f8-4a95-a740-010dcf56cc90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18e41a3e7f2147329cfc9a9f7b0c1036', 'request_id': 'req-00666838-14f8-4a95-a740-010dcf56cc90'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:30:02.739600 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-00666838-14f8-4a95-a740-010dcf56cc90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18e41a3e7f2147329cfc9a9f7b0c1036', 'request_id': 'req-00666838-14f8-4a95-a740-010dcf56cc90'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:30:02.747383 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-00666838-14f8-4a95-a740-010dcf56cc90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18e41a3e7f2147329cfc9a9f7b0c1036', 'request_id': 'req-00666838-14f8-4a95-a740-010dcf56cc90'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:30:02.757709 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-00666838-14f8-4a95-a740-010dcf56cc90 admin admin] The token cache is being invalidate because project 18e41a3e7f2147329cfc9a9f7b0c1036 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:30:02.757994 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-00666838-14f8-4a95-a740-010dcf56cc90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:30:02.758623 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-00666838-14f8-4a95-a740-010dcf56cc90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:30:02.760217 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1452/2896] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:30:02 2022] DELETE /identity/v3/projects/18e41a3e7f2147329cfc9a9f7b0c1036 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:30:12.388443 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=61,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:30:12.754617 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=33,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:31:23.222947 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ada2c3-cbac-4f28-8630-095f989414f0 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:31:23.224506 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-54ada2c3-cbac-4f28-8630-095f989414f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:31:23.236076 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ada2c3-cbac-4f28-8630-095f989414f0 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:31:23.287840 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b011a58e-6459-4736-8965-137aa4da2478 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:31:23.288478 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011a58e-6459-4736-8965-137aa4da2478 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:31:23.288755 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011a58e-6459-4736-8965-137aa4da2478 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:31:23.289059 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011a58e-6459-4736-8965-137aa4da2478 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:31:23.290839 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b011a58e-6459-4736-8965-137aa4da2478 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:31:23.366513 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b011a58e-6459-4736-8965-137aa4da2478 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:31:23.368037 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b011a58e-6459-4736-8965-137aa4da2478 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:31:23.379643 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1445/2897] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:31:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:31:23.966279 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00666838-14f8-4a95-a740-010dcf56cc90 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:31:23.968251 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-00666838-14f8-4a95-a740-010dcf56cc90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:31:23.989782 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00666838-14f8-4a95-a740-010dcf56cc90 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:31:23.996216 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b1e2f-4db4-44ab-ac2d-6ecbb509fa1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:31:23.997206 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b1e2f-4db4-44ab-ac2d-6ecbb509fa1a service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:31:23.997639 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b1e2f-4db4-44ab-ac2d-6ecbb509fa1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:31:23.998024 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b1e2f-4db4-44ab-ac2d-6ecbb509fa1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:31:24.000169 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-f36b1e2f-4db4-44ab-ac2d-6ecbb509fa1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:31:24.012434 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b1e2f-4db4-44ab-ac2d-6ecbb509fa1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:31:24.014668 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b1e2f-4db4-44ab-ac2d-6ecbb509fa1a service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:31:24.026202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1453/2898] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:31:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:31:29.200642 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b011a58e-6459-4736-8965-137aa4da2478 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:31:29.202088 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b011a58e-6459-4736-8965-137aa4da2478 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:31:29.213485 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b011a58e-6459-4736-8965-137aa4da2478 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:31:29.217171 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bdda2a-64c9-4a06-aaf3-b94b6a66234a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:31:29.217884 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bdda2a-64c9-4a06-aaf3-b94b6a66234a service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:31:29.218078 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bdda2a-64c9-4a06-aaf3-b94b6a66234a service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:31:29.218361 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bdda2a-64c9-4a06-aaf3-b94b6a66234a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:31:29.220106 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-93bdda2a-64c9-4a06-aaf3-b94b6a66234a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:31:29.233564 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bdda2a-64c9-4a06-aaf3-b94b6a66234a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:31:29.234990 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bdda2a-64c9-4a06-aaf3-b94b6a66234a service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:31:29.245922 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1446/2899] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:31:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:31:30.479001 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b1e2f-4db4-44ab-ac2d-6ecbb509fa1a service neutron] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:31:30.480638 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-f36b1e2f-4db4-44ab-ac2d-6ecbb509fa1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:31:30.497579 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b1e2f-4db4-44ab-ac2d-6ecbb509fa1a service neutron] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:31:30.503288 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bdf537-f5d8-4a1a-9773-a8b83428f2e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:31:30.503985 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bdf537-f5d8-4a1a-9773-a8b83428f2e3 service nova] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:31:30.504274 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bdf537-f5d8-4a1a-9773-a8b83428f2e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:31:30.504557 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bdf537-f5d8-4a1a-9773-a8b83428f2e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:31:30.506361 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-88bdf537-f5d8-4a1a-9773-a8b83428f2e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:31:30.520104 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bdf537-f5d8-4a1a-9773-a8b83428f2e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:31:30.521666 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bdf537-f5d8-4a1a-9773-a8b83428f2e3 service nova] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:31:30.532961 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1454/2900] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:31:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:31:31.553596 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88943163-159f-4b4e-9a20-d78ea7bfb017 None None] REQUEST_METHOD: `POST` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:31:31.554058 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88943163-159f-4b4e-9a20-d78ea7bfb017 None None] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:31:31.554174 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88943163-159f-4b4e-9a20-d78ea7bfb017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:31:31.572112 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.auth.core [None req-88943163-159f-4b4e-9a20-d78ea7bfb017 None None] MFA Rules not processed for user `a32702a67a7b49ffa153fc59251fd169`. Rule list: `[]` (Enabled: `True`). {{(pid=80284) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 23:31:31.573318 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-88943163-159f-4b4e-9a20-d78ea7bfb017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:31:31.626601 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1447/2901] 10.222.0.60 () {58 vars in 967 bytes} [Wed Jul 13 23:31:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 23:31:31.667499 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bdf537-f5d8-4a1a-9773-a8b83428f2e3 service nova] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:31:31.668970 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-88bdf537-f5d8-4a1a-9773-a8b83428f2e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:31:31.737527 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bdf537-f5d8-4a1a-9773-a8b83428f2e3 service nova] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:31:31.742597 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7f6a13-5f8b-423d-9f53-f63cddc93f6d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '786fc44611804d3ca6280ffcab415569', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:31:31.743245 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f6a13-5f8b-423d-9f53-f63cddc93f6d service swift] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:31:31.743549 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f6a13-5f8b-423d-9f53-f63cddc93f6d service swift] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:31:31.743840 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f6a13-5f8b-423d-9f53-f63cddc93f6d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:31:31.751992 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7f6a13-5f8b-423d-9f53-f63cddc93f6d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:31:31.753587 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7f6a13-5f8b-423d-9f53-f63cddc93f6d service swift] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:31:31.762176 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1455/2902] 10.222.0.60 () {60 vars in 1384 bytes} [Wed Jul 13 23:31:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 23:31:38.254579 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88943163-159f-4b4e-9a20-d78ea7bfb017 None None] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:31:38.256525 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-88943163-159f-4b4e-9a20-d78ea7bfb017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:31:38.298472 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=100 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:31:38.334808 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88943163-159f-4b4e-9a20-d78ea7bfb017 None None] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:31:38.339787 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ed035-ee53-43b8-a404-3ad0e2667439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:31:38.340686 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ed035-ee53-43b8-a404-3ad0e2667439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:31:38.340960 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ed035-ee53-43b8-a404-3ad0e2667439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:31:38.341232 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ed035-ee53-43b8-a404-3ad0e2667439 admin admin] PATH_INFO: `/v3/users/0780487ed8534aa0846d6c8cd8522ada` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:31:38.342615 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135ed035-ee53-43b8-a404-3ad0e2667439 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0780487ed8534aa0846d6c8cd8522ada)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:31:38.344665 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135ed035-ee53-43b8-a404-3ad0e2667439 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:31:38.406008 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-135ed035-ee53-43b8-a404-3ad0e2667439 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0780487ed8534aa0846d6c8cd8522ada'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:31:38.406806 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-135ed035-ee53-43b8-a404-3ad0e2667439 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0780487ed8534aa0846d6c8cd8522ada'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:31:38.426241 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-135ed035-ee53-43b8-a404-3ad0e2667439 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0780487ed8534aa0846d6c8cd8522ada'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:31:38.437644 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-135ed035-ee53-43b8-a404-3ad0e2667439 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0780487ed8534aa0846d6c8cd8522ada'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:31:38.451200 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-135ed035-ee53-43b8-a404-3ad0e2667439 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0780487ed8534aa0846d6c8cd8522ada'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:31:38.459085 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1448/2903] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:31:38 2022] DELETE /identity/v3/users/0780487ed8534aa0846d6c8cd8522ada => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:31:38.659667 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7f6a13-5f8b-423d-9f53-f63cddc93f6d service swift] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:31:38.672262 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7f6a13-5f8b-423d-9f53-f63cddc93f6d service swift] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:31:38.738711 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f80f4-9258-4a89-9440-be7a05dd2b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:31:38.739622 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f80f4-9258-4a89-9440-be7a05dd2b39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:31:38.739906 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f80f4-9258-4a89-9440-be7a05dd2b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:31:38.740202 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f80f4-9258-4a89-9440-be7a05dd2b39 admin admin] PATH_INFO: `/v3/projects/f96223da0efe45daaf57326c8f9d1ae1` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:31:38.741409 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52f80f4-9258-4a89-9440-be7a05dd2b39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f96223da0efe45daaf57326c8f9d1ae1)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:31:38.743609 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52f80f4-9258-4a89-9440-be7a05dd2b39 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:31:38.800234 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-a52f80f4-9258-4a89-9440-be7a05dd2b39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f96223da0efe45daaf57326c8f9d1ae1', 'request_id': 'req-a52f80f4-9258-4a89-9440-be7a05dd2b39'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:31:38.809560 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-a52f80f4-9258-4a89-9440-be7a05dd2b39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f96223da0efe45daaf57326c8f9d1ae1', 'request_id': 'req-a52f80f4-9258-4a89-9440-be7a05dd2b39'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:31:38.819826 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-a52f80f4-9258-4a89-9440-be7a05dd2b39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f96223da0efe45daaf57326c8f9d1ae1', 'request_id': 'req-a52f80f4-9258-4a89-9440-be7a05dd2b39'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:31:38.826578 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:31:38.839586 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-a52f80f4-9258-4a89-9440-be7a05dd2b39 admin admin] The token cache is being invalidate because project f96223da0efe45daaf57326c8f9d1ae1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:31:38.840173 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-a52f80f4-9258-4a89-9440-be7a05dd2b39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:31:38.841415 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-a52f80f4-9258-4a89-9440-be7a05dd2b39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:31:38.843983 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1456/2904] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:31:38 2022] DELETE /identity/v3/projects/f96223da0efe45daaf57326c8f9d1ae1 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:31:38.849468 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ed035-ee53-43b8-a404-3ad0e2667439 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:31:38.850996 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-135ed035-ee53-43b8-a404-3ad0e2667439 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:31:38.865322 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ed035-ee53-43b8-a404-3ad0e2667439 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:31:38.933752 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15fec29-74df-46f3-92fd-e2123c665467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:31:38.934686 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fec29-74df-46f3-92fd-e2123c665467 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:31:38.934968 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fec29-74df-46f3-92fd-e2123c665467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:31:38.935254 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fec29-74df-46f3-92fd-e2123c665467 admin admin] PATH_INFO: `/v3/users/1f2117c90756499fb687291da7a6642c` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:31:38.937070 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15fec29-74df-46f3-92fd-e2123c665467 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f2117c90756499fb687291da7a6642c)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:31:38.939034 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15fec29-74df-46f3-92fd-e2123c665467 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:31:39.000857 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f15fec29-74df-46f3-92fd-e2123c665467 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f2117c90756499fb687291da7a6642c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:31:39.001558 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f15fec29-74df-46f3-92fd-e2123c665467 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f2117c90756499fb687291da7a6642c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:31:39.007233 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:31:39.015788 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f15fec29-74df-46f3-92fd-e2123c665467 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f2117c90756499fb687291da7a6642c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:31:39.023255 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f15fec29-74df-46f3-92fd-e2123c665467 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f2117c90756499fb687291da7a6642c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:31:39.032142 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-f15fec29-74df-46f3-92fd-e2123c665467 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f2117c90756499fb687291da7a6642c'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:31:39.038832 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1449/2905] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:31:38 2022] DELETE /identity/v3/users/1f2117c90756499fb687291da7a6642c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:31:39.227209 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f80f4-9258-4a89-9440-be7a05dd2b39 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:31:39.228877 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-a52f80f4-9258-4a89-9440-be7a05dd2b39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:31:39.242662 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f80f4-9258-4a89-9440-be7a05dd2b39 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:31:39.311523 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7078d10c-8b17-47d2-8807-f2d1fb7d6e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:31:39.312697 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078d10c-8b17-47d2-8807-f2d1fb7d6e72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:31:39.312983 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078d10c-8b17-47d2-8807-f2d1fb7d6e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:31:39.313268 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078d10c-8b17-47d2-8807-f2d1fb7d6e72 admin admin] PATH_INFO: `/v3/projects/83937178a24e4636a96243d0fbcd67b5` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:31:39.314506 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7078d10c-8b17-47d2-8807-f2d1fb7d6e72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83937178a24e4636a96243d0fbcd67b5)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:31:39.316478 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7078d10c-8b17-47d2-8807-f2d1fb7d6e72 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:31:39.368310 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-7078d10c-8b17-47d2-8807-f2d1fb7d6e72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83937178a24e4636a96243d0fbcd67b5', 'request_id': 'req-7078d10c-8b17-47d2-8807-f2d1fb7d6e72'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:31:39.374687 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-7078d10c-8b17-47d2-8807-f2d1fb7d6e72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83937178a24e4636a96243d0fbcd67b5', 'request_id': 'req-7078d10c-8b17-47d2-8807-f2d1fb7d6e72'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:31:39.382900 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-7078d10c-8b17-47d2-8807-f2d1fb7d6e72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83937178a24e4636a96243d0fbcd67b5', 'request_id': 'req-7078d10c-8b17-47d2-8807-f2d1fb7d6e72'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:31:39.394347 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-7078d10c-8b17-47d2-8807-f2d1fb7d6e72 admin admin] The token cache is being invalidate because project 83937178a24e4636a96243d0fbcd67b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80285) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:31:39.394564 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-7078d10c-8b17-47d2-8807-f2d1fb7d6e72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:31:39.395215 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-7078d10c-8b17-47d2-8807-f2d1fb7d6e72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:31:39.396807 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1457/2906] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:31:39 2022] DELETE /identity/v3/projects/83937178a24e4636a96243d0fbcd67b5 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:31:49.034867 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=10,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:31:49.391048 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:DELETE=8,keystone:SELECT=53,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:32:05.561382 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15fec29-74df-46f3-92fd-e2123c665467 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:32:05.562911 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-f15fec29-74df-46f3-92fd-e2123c665467 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:32:05.618980 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15fec29-74df-46f3-92fd-e2123c665467 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:32:05.665741 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5913c709-42b7-4902-b145-cb8d2a966496 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:32:05.666379 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913c709-42b7-4902-b145-cb8d2a966496 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:32:05.666904 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913c709-42b7-4902-b145-cb8d2a966496 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:32:05.667176 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913c709-42b7-4902-b145-cb8d2a966496 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:32:05.669531 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-5913c709-42b7-4902-b145-cb8d2a966496 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:32:05.678695 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5913c709-42b7-4902-b145-cb8d2a966496 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:32:05.680149 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5913c709-42b7-4902-b145-cb8d2a966496 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:32:05.688015 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1450/2907] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:32:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:32:06.794616 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7078d10c-8b17-47d2-8807-f2d1fb7d6e72 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:32:06.796116 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-7078d10c-8b17-47d2-8807-f2d1fb7d6e72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:32:06.808984 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7078d10c-8b17-47d2-8807-f2d1fb7d6e72 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:32:06.813258 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e739c1b1-8475-44a9-9478-8ffc5844e59d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:32:06.813944 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739c1b1-8475-44a9-9478-8ffc5844e59d service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:32:06.814218 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739c1b1-8475-44a9-9478-8ffc5844e59d service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:32:06.814569 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739c1b1-8475-44a9-9478-8ffc5844e59d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:32:06.816442 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-e739c1b1-8475-44a9-9478-8ffc5844e59d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:32:06.827006 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e739c1b1-8475-44a9-9478-8ffc5844e59d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:32:06.828552 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e739c1b1-8475-44a9-9478-8ffc5844e59d service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:32:06.837956 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1458/2908] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:32:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:32:06.941707 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5913c709-42b7-4902-b145-cb8d2a966496 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:32:06.950353 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5913c709-42b7-4902-b145-cb8d2a966496 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:32:06.954169 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9502a9a-9c31-49b2-adaa-939dbf943395 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:32:06.954798 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9502a9a-9c31-49b2-adaa-939dbf943395 service neutron] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:32:06.955081 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9502a9a-9c31-49b2-adaa-939dbf943395 service neutron] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:32:06.955350 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9502a9a-9c31-49b2-adaa-939dbf943395 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:32:06.957157 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b9502a9a-9c31-49b2-adaa-939dbf943395 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:32:07.031277 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9502a9a-9c31-49b2-adaa-939dbf943395 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:32:07.032682 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9502a9a-9c31-49b2-adaa-939dbf943395 service neutron] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:32:07.044544 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1451/2909] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:32:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:32:07.107100 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e739c1b1-8475-44a9-9478-8ffc5844e59d service neutron] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:32:07.108577 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-e739c1b1-8475-44a9-9478-8ffc5844e59d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:32:07.177664 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e739c1b1-8475-44a9-9478-8ffc5844e59d service neutron] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:32:07.183120 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ea4fb-707a-45cd-9684-ba38ddddb12a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64824ba84a024930bbc177f8a8d9a046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:32:07.183805 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ea4fb-707a-45cd-9684-ba38ddddb12a service placement] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:32:07.184093 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ea4fb-707a-45cd-9684-ba38ddddb12a service placement] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:32:07.184410 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ea4fb-707a-45cd-9684-ba38ddddb12a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:32:07.186190 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-b59ea4fb-707a-45cd-9684-ba38ddddb12a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:32:07.197660 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ea4fb-707a-45cd-9684-ba38ddddb12a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:32:07.199252 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ea4fb-707a-45cd-9684-ba38ddddb12a service placement] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:32:07.209305 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1459/2910] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:32:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:32:17.038786 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=80 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:32:17.204980 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=38 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:33:35.170049 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9502a9a-9c31-49b2-adaa-939dbf943395 service neutron] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:33:35.171458 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-b9502a9a-9c31-49b2-adaa-939dbf943395 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:33:35.244331 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9502a9a-9c31-49b2-adaa-939dbf943395 service neutron] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:33:35.250288 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb5a54-275e-450d-a322-ddad28573c76 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64824ba84a024930bbc177f8a8d9a046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:33:35.250933 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb5a54-275e-450d-a322-ddad28573c76 service placement] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:33:35.251743 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb5a54-275e-450d-a322-ddad28573c76 service placement] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:33:35.252019 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb5a54-275e-450d-a322-ddad28573c76 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:33:35.253812 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8fbb5a54-275e-450d-a322-ddad28573c76 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:33:35.266226 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb5a54-275e-450d-a322-ddad28573c76 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:33:35.268501 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb5a54-275e-450d-a322-ddad28573c76 service placement] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:33:35.279182 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1452/2911] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:33:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:33:45.274586 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=31 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:33:51.163165 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ea4fb-707a-45cd-9684-ba38ddddb12a service placement] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:33:51.179861 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ea4fb-707a-45cd-9684-ba38ddddb12a service placement] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:33:51.183432 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29764530-fd42-43ab-a58f-19b270114cba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:33:51.184110 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29764530-fd42-43ab-a58f-19b270114cba service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:33:51.184394 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29764530-fd42-43ab-a58f-19b270114cba service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:33:51.184672 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29764530-fd42-43ab-a58f-19b270114cba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:33:51.186444 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-29764530-fd42-43ab-a58f-19b270114cba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:33:51.245606 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29764530-fd42-43ab-a58f-19b270114cba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:33:51.247057 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29764530-fd42-43ab-a58f-19b270114cba service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:33:51.256141 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1460/2912] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:33:51 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:33:52.341580 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb5a54-275e-450d-a322-ddad28573c76 service placement] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:33:52.343399 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-8fbb5a54-275e-450d-a322-ddad28573c76 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:33:52.368210 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb5a54-275e-450d-a322-ddad28573c76 service placement] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:33:52.373753 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebbe8e6-52a0-47f5-916d-58157ebd1bdd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4789384d158b4995bd30e1e37e30f5f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:33:52.374407 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbe8e6-52a0-47f5-916d-58157ebd1bdd service nova] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:33:52.374692 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbe8e6-52a0-47f5-916d-58157ebd1bdd service nova] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:33:52.374973 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbe8e6-52a0-47f5-916d-58157ebd1bdd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:33:52.376868 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-2ebbe8e6-52a0-47f5-916d-58157ebd1bdd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:33:52.390992 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebbe8e6-52a0-47f5-916d-58157ebd1bdd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:33:52.392655 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebbe8e6-52a0-47f5-916d-58157ebd1bdd service nova] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:33:52.405343 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1453/2913] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:33:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:33:56.143842 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29764530-fd42-43ab-a58f-19b270114cba service neutron] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:33:56.145278 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-29764530-fd42-43ab-a58f-19b270114cba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:33:56.156436 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29764530-fd42-43ab-a58f-19b270114cba service neutron] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:33:56.160653 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db177ff-9228-4f49-96b1-d5bc52480be0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d670df2aff5478ebf61f55358d554de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:33:56.161301 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db177ff-9228-4f49-96b1-d5bc52480be0 service neutron] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:33:56.161582 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db177ff-9228-4f49-96b1-d5bc52480be0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:33:56.161854 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db177ff-9228-4f49-96b1-d5bc52480be0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:33:56.163664 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-5db177ff-9228-4f49-96b1-d5bc52480be0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:33:56.226110 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db177ff-9228-4f49-96b1-d5bc52480be0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:33:56.228215 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db177ff-9228-4f49-96b1-d5bc52480be0 service neutron] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:33:56.240479 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1461/2914] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:33:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:33:59.289896 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebbe8e6-52a0-47f5-916d-58157ebd1bdd service nova] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:33:59.302249 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebbe8e6-52a0-47f5-916d-58157ebd1bdd service nova] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:33:59.307325 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e517a3e4-9d94-4afb-8727-b2a884a23f15 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64824ba84a024930bbc177f8a8d9a046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:33:59.308004 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517a3e4-9d94-4afb-8727-b2a884a23f15 service placement] REQUEST_METHOD: `GET` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:33:59.308284 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517a3e4-9d94-4afb-8727-b2a884a23f15 service placement] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:33:59.308562 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517a3e4-9d94-4afb-8727-b2a884a23f15 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:33:59.310335 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-e517a3e4-9d94-4afb-8727-b2a884a23f15 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:33:59.323001 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e517a3e4-9d94-4afb-8727-b2a884a23f15 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:33:59.324695 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e517a3e4-9d94-4afb-8727-b2a884a23f15 service placement] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:33:59.335292 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1454/2915] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:33:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:34:02.261104 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db177ff-9228-4f49-96b1-d5bc52480be0 service neutron] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:34:02.262924 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-5db177ff-9228-4f49-96b1-d5bc52480be0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:34:02.276381 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db177ff-9228-4f49-96b1-d5bc52480be0 service neutron] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:34:02.281061 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06520db-18d0-44e0-8101-5fd6bec76ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:34:02.282506 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06520db-18d0-44e0-8101-5fd6bec76ea8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:34:02.282817 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06520db-18d0-44e0-8101-5fd6bec76ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:34:02.283093 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06520db-18d0-44e0-8101-5fd6bec76ea8 admin admin] PATH_INFO: `/v3/users/e292de524b9c405dae3d520b60136155` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:34:02.284331 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06520db-18d0-44e0-8101-5fd6bec76ea8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e292de524b9c405dae3d520b60136155)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:34:02.286288 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06520db-18d0-44e0-8101-5fd6bec76ea8 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:34:02.336765 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e06520db-18d0-44e0-8101-5fd6bec76ea8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e292de524b9c405dae3d520b60136155'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:34:02.337429 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e06520db-18d0-44e0-8101-5fd6bec76ea8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e292de524b9c405dae3d520b60136155'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:34:02.350510 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e06520db-18d0-44e0-8101-5fd6bec76ea8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e292de524b9c405dae3d520b60136155'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:34:02.357716 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e06520db-18d0-44e0-8101-5fd6bec76ea8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e292de524b9c405dae3d520b60136155'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:34:02.358447 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-e06520db-18d0-44e0-8101-5fd6bec76ea8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e292de524b9c405dae3d520b60136155'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:34:02.370312 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:34:02.377091 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1462/2916] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:34:02 2022] DELETE /identity/v3/users/e292de524b9c405dae3d520b60136155 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 23:34:02.562388 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e517a3e4-9d94-4afb-8727-b2a884a23f15 service placement] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:34:02.573138 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e517a3e4-9d94-4afb-8727-b2a884a23f15 service placement] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:34:02.633291 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39309569-10d2-4a90-805d-aff6a25cf696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:34:02.634153 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39309569-10d2-4a90-805d-aff6a25cf696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:34:02.634437 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39309569-10d2-4a90-805d-aff6a25cf696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:34:02.634710 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39309569-10d2-4a90-805d-aff6a25cf696 admin admin] PATH_INFO: `/v3/projects/d2f20deb401b4c348b0a18177ec0d2d7` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:34:02.635929 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39309569-10d2-4a90-805d-aff6a25cf696 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2f20deb401b4c348b0a18177ec0d2d7)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:34:02.637829 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39309569-10d2-4a90-805d-aff6a25cf696 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:34:02.685677 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-39309569-10d2-4a90-805d-aff6a25cf696 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2f20deb401b4c348b0a18177ec0d2d7', 'request_id': 'req-39309569-10d2-4a90-805d-aff6a25cf696'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:34:02.692119 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-39309569-10d2-4a90-805d-aff6a25cf696 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2f20deb401b4c348b0a18177ec0d2d7', 'request_id': 'req-39309569-10d2-4a90-805d-aff6a25cf696'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:34:02.700374 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-39309569-10d2-4a90-805d-aff6a25cf696 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2f20deb401b4c348b0a18177ec0d2d7', 'request_id': 'req-39309569-10d2-4a90-805d-aff6a25cf696'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:34:02.711270 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-39309569-10d2-4a90-805d-aff6a25cf696 admin admin] The token cache is being invalidate because project d2f20deb401b4c348b0a18177ec0d2d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:34:02.711578 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-39309569-10d2-4a90-805d-aff6a25cf696 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:34:02.712203 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-39309569-10d2-4a90-805d-aff6a25cf696 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:34:02.713725 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1455/2917] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:34:02 2022] DELETE /identity/v3/projects/d2f20deb401b4c348b0a18177ec0d2d7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:34:02.717801 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06520db-18d0-44e0-8101-5fd6bec76ea8 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:34:02.719270 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-e06520db-18d0-44e0-8101-5fd6bec76ea8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:34:02.732419 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06520db-18d0-44e0-8101-5fd6bec76ea8 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:34:02.788453 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5480fc-2588-41bb-b13f-2db32cef3727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:34:02.789333 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5480fc-2588-41bb-b13f-2db32cef3727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:34:02.789619 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5480fc-2588-41bb-b13f-2db32cef3727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:34:02.789897 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5480fc-2588-41bb-b13f-2db32cef3727 admin admin] PATH_INFO: `/v3/users/9440975fa183447bb404c1ea01117c61` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:34:02.791348 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5480fc-2588-41bb-b13f-2db32cef3727 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9440975fa183447bb404c1ea01117c61)` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:34:02.793336 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5480fc-2588-41bb-b13f-2db32cef3727 admin admin] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:34:02.839423 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-9a5480fc-2588-41bb-b13f-2db32cef3727 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9440975fa183447bb404c1ea01117c61'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:34:02.840108 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-9a5480fc-2588-41bb-b13f-2db32cef3727 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9440975fa183447bb404c1ea01117c61'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:34:02.853249 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-9a5480fc-2588-41bb-b13f-2db32cef3727 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9440975fa183447bb404c1ea01117c61'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:34:02.860531 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-9a5480fc-2588-41bb-b13f-2db32cef3727 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9440975fa183447bb404c1ea01117c61'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:34:02.861258 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.notifications [None req-9a5480fc-2588-41bb-b13f-2db32cef3727 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9440975fa183447bb404c1ea01117c61'} {{(pid=80285) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:34:02.874181 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1463/2918] 10.222.0.60 () {58 vars in 1223 bytes} [Wed Jul 13 23:34:02 2022] DELETE /identity/v3/users/9440975fa183447bb404c1ea01117c61 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:34:03.051886 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39309569-10d2-4a90-805d-aff6a25cf696 admin admin] Authenticating user token {{(pid=80284) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:34:03.053338 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.fernet_utils [None req-39309569-10d2-4a90-805d-aff6a25cf696 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80284) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:34:03.071595 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39309569-10d2-4a90-805d-aff6a25cf696 admin admin] Validating token access rules against request {{(pid=80284) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:34:03.128273 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ca27a-75d6-428e-b474-a4f2d3340802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f1cf8ea49bb4e7fb83c77a4a463d6fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c5ffbcb80ba6476698f912924febd384', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80284) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:34:03.129205 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ca27a-75d6-428e-b474-a4f2d3340802 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:34:03.129491 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ca27a-75d6-428e-b474-a4f2d3340802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:34:03.129766 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ca27a-75d6-428e-b474-a4f2d3340802 admin admin] PATH_INFO: `/v3/projects/c13ca5d8f19349e19431c40f3bae5893` {{(pid=80284) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:34:03.130974 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505ca27a-75d6-428e-b474-a4f2d3340802 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c13ca5d8f19349e19431c40f3bae5893)` {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:34:03.132921 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505ca27a-75d6-428e-b474-a4f2d3340802 admin admin] RBAC: Authorization granted {{(pid=80284) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:34:03.139663 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:34:03.181064 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-505ca27a-75d6-428e-b474-a4f2d3340802 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c13ca5d8f19349e19431c40f3bae5893', 'request_id': 'req-505ca27a-75d6-428e-b474-a4f2d3340802'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:34:03.187004 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-505ca27a-75d6-428e-b474-a4f2d3340802 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c13ca5d8f19349e19431c40f3bae5893', 'request_id': 'req-505ca27a-75d6-428e-b474-a4f2d3340802'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:34:03.194699 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-505ca27a-75d6-428e-b474-a4f2d3340802 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c13ca5d8f19349e19431c40f3bae5893', 'request_id': 'req-505ca27a-75d6-428e-b474-a4f2d3340802'} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:34:03.204798 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-505ca27a-75d6-428e-b474-a4f2d3340802 admin admin] The token cache is being invalidate because project c13ca5d8f19349e19431c40f3bae5893 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80284) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 23:34:03.205082 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-505ca27a-75d6-428e-b474-a4f2d3340802 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:34:03.205705 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG keystone.notifications [None req-505ca27a-75d6-428e-b474-a4f2d3340802 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80284) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 23:34:03.207368 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: [pid: 80284|app: 0|req: 1456/2919] 10.222.0.60 () {58 vars in 1232 bytes} [Wed Jul 13 23:34:03 2022] DELETE /identity/v3/projects/c13ca5d8f19349e19431c40f3bae5893 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 23:34:12.870585 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=54,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:34:13.201754 dv-cinder-799871-29-iscsi devstack@keystone.service[80284]: DEBUG dbcounter [-] [80284] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80284) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 23:38:43.179122 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5480fc-2588-41bb-b13f-2db32cef3727 admin admin] Authenticating user token {{(pid=80285) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 23:38:43.180555 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-9a5480fc-2588-41bb-b13f-2db32cef3727 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:38:43.259751 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5480fc-2588-41bb-b13f-2db32cef3727 admin admin] Validating token access rules against request {{(pid=80285) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 23:38:43.319254 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d09914-2a53-4e02-8fae-622609014ff5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64824ba84a024930bbc177f8a8d9a046', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab0a5581e4e444c293815e1304e20c05', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80285) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 23:38:43.319934 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d09914-2a53-4e02-8fae-622609014ff5 service placement] REQUEST_METHOD: `GET` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 23:38:43.320221 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d09914-2a53-4e02-8fae-622609014ff5 service placement] SCRIPT_NAME: `/identity` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 23:38:43.320501 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d09914-2a53-4e02-8fae-622609014ff5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80285) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 23:38:43.322302 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.fernet_utils [None req-a0d09914-2a53-4e02-8fae-622609014ff5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80285) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 23:38:43.334161 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d09914-2a53-4e02-8fae-622609014ff5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 23:38:43.343323 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d09914-2a53-4e02-8fae-622609014ff5 service placement] RBAC: Authorization granted {{(pid=80285) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 23:38:43.354185 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: [pid: 80285|app: 0|req: 1464/2920] 10.222.0.60 () {60 vars in 1365 bytes} [Wed Jul 13 23:38:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 23:38:53.349856 dv-cinder-799871-29-iscsi devstack@keystone.service[80285]: DEBUG dbcounter [-] [80285] Writing DB stats keystone:SELECT=64 {{(pid=80285) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}